[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:05 dist-test-slave-g170 krb5kdc[24348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:05 dist-test-slave-g170 krb5kdc[24348](info): set up 2 sockets
Aug 05 22:42:05 dist-test-slave-g170 krb5kdc[24348](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260805 22:42:06.377046 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:06 dist-test-slave-g170 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:06.509599 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33215
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:33215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:06.597082 24369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:06.597350 24369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:06.597393 24369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:06.598650 24369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:06.598731 24369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:06.598784 24369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:06.598815 24369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:06.598850 24369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:06.600549 24369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:33215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:06.601012 24369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:06.601310 24369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:06.608505 24378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:06.608508 24382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:06.609004 24377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:06.608966 24369 server_base.cc:1034] running on GCE node
I20260805 22:42:06.609385 24369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:06.609850 24369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:06.611030 24369 hybrid_clock.cc:648] HybridClock initialized: now 1785969726611003 us; error 82 us; skew 500 ppm
Aug 05 22:42:06 dist-test-slave-g170 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969726, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:06.613955 24369 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:06.614610 24369 webserver.cc:460] Webserver started at http://127.23.197.126:41945/ using document root <none> and password file <none>
I20260805 22:42:06.614936 24369 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:06.614989 24369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:06.615238 24369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:06.616453 24369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "6aaad2662b1d40a49a05295ef185c2ce"
format_stamp: "Formatted at 2026-08-05 22:42:06 on dist-test-slave-g170"
server_key: "855d30139a03420022c0df3aacf4dcb2"
server_key_iv: "514fa0c92455fbdc12f81303a2284717"
server_key_version: "encryptionkey@0"
I20260805 22:42:06.616894 24369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "6aaad2662b1d40a49a05295ef185c2ce"
format_stamp: "Formatted at 2026-08-05 22:42:06 on dist-test-slave-g170"
server_key: "855d30139a03420022c0df3aacf4dcb2"
server_key_iv: "514fa0c92455fbdc12f81303a2284717"
server_key_version: "encryptionkey@0"
I20260805 22:42:06.618896 24369 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:06.620023 24393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:06.620393 24369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:06.620512 24369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "6aaad2662b1d40a49a05295ef185c2ce"
format_stamp: "Formatted at 2026-08-05 22:42:06 on dist-test-slave-g170"
server_key: "855d30139a03420022c0df3aacf4dcb2"
server_key_iv: "514fa0c92455fbdc12f81303a2284717"
server_key_version: "encryptionkey@0"
I20260805 22:42:06.620591 24369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:06.646921 24369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:06.649636 24369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:06.649855 24369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:06.654925 24369 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:33215
I20260805 22:42:06.655157 24458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:33215 every 8 connection(s)
I20260805 22:42:06.655630 24369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:06.657137 24459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:06.660313 24459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce: Bootstrap starting.
I20260805 22:42:06.661262 24459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:06.661652 24459 log.cc:824] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:06.662535 24459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce: No bootstrap required, opened a new log
I20260805 22:42:06.664383 24459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aaad2662b1d40a49a05295ef185c2ce" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33215 } }
I20260805 22:42:06.664548 24459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:06.664640 24459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aaad2662b1d40a49a05295ef185c2ce, State: Initialized, Role: FOLLOWER
I20260805 22:42:06.664878 24459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aaad2662b1d40a49a05295ef185c2ce" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33215 } }
I20260805 22:42:06.665015 24459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:06.665042 24459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:06.665119 24459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:06.665660 24459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aaad2662b1d40a49a05295ef185c2ce" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33215 } }
I20260805 22:42:06.665825 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24369
I20260805 22:42:06.665885 24459 leader_election.cc:302] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6aaad2662b1d40a49a05295ef185c2ce; no voters:
I20260805 22:42:06.665935 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:06.666126 24459 leader_election.cc:288] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:06.666177 24464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:06.666384 24464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [term 1 LEADER]: Becoming Leader. State: Replica: 6aaad2662b1d40a49a05295ef185c2ce, State: Running, Role: LEADER
I20260805 22:42:06.666563 24464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aaad2662b1d40a49a05295ef185c2ce" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33215 } }
I20260805 22:42:06.666683 24459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:06.666824 24341 external_mini_cluster.cc:1428] Setting key af771a39b029682a08eaf51086def698
I20260805 22:42:06.667161 24464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aaad2662b1d40a49a05295ef185c2ce. Latest consensus state: current_term: 1 leader_uuid: "6aaad2662b1d40a49a05295ef185c2ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aaad2662b1d40a49a05295ef185c2ce" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33215 } } }
I20260805 22:42:06.667250 24464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:06.667424 24465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6aaad2662b1d40a49a05295ef185c2ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aaad2662b1d40a49a05295ef185c2ce" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33215 } } }
I20260805 22:42:06.667503 24465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:06.667822 24478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:06.669685 24478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:06.674256 24478 catalog_manager.cc:1347] Generated new cluster ID: 7d828a80862442f7a25de35146965ec8
I20260805 22:42:06.674310 24478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:42:06 dist-test-slave-g170 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:06.683750 24478 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:06.684628 24478 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:06.689167 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:06.668888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.23.197.126:33215)
0805 22:42:06.669151 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:06.669161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:06.669198 (+ 37us) server_negotiation.cc:407] Connection header received
0805 22:42:06.671388 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:06.671397 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:06.671510 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:06.671657 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:06.673296 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:06.674882 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:06.676845 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:06.677640 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:06.677757 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:06.684108 (+ 6351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:06.684112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:06.684112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:06.684123 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:06.686784 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:06.687340 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:06.687343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:06.687344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:06.687446 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:06.687808 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:06.687808 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:06.687809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:06.688062 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:06.688347 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:06.688580 (+ 233us) server_negotiation.cc:299] Negotiation successful
0805 22:42:06.688789 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260805 22:42:06.691608 24478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6aaad2662b1d40a49a05295ef185c2ce: Generated new TSK 0
I20260805 22:42:06.691838 24478 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:06.724387 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33215
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:06.835052 24504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:06.835269 24504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:06.835314 24504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:06.836560 24504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:06.836619 24504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:06.836648 24504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:06.838279 24504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33215
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:06.838635 24504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:06.838915 24504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:06.846304 24510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:06.846381 24509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:06.846345 24512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:06.846606 24504 server_base.cc:1034] running on GCE node
I20260805 22:42:06.846890 24504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:06.847162 24504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:06.848356 24504 hybrid_clock.cc:648] HybridClock initialized: now 1785969726848327 us; error 49 us; skew 500 ppm
Aug 05 22:42:06 dist-test-slave-g170 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969726, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:06.852149 24504 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:06.853039 24504 webserver.cc:460] Webserver started at http://127.23.197.65:34797/ using document root <none> and password file <none>
I20260805 22:42:06.853615 24504 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:06.853776 24504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:06.853971 24504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:06.855868 24504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "de7f3010b9274821a34c04af984b9e87"
format_stamp: "Formatted at 2026-08-05 22:42:06 on dist-test-slave-g170"
server_key: "1b90b452c48cdb8371a32a8fe1bd12c7"
server_key_iv: "5d7458d1f5b9a96a9d4e990f20f2d136"
server_key_version: "encryptionkey@0"
I20260805 22:42:06.856583 24504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "de7f3010b9274821a34c04af984b9e87"
format_stamp: "Formatted at 2026-08-05 22:42:06 on dist-test-slave-g170"
server_key: "1b90b452c48cdb8371a32a8fe1bd12c7"
server_key_iv: "5d7458d1f5b9a96a9d4e990f20f2d136"
server_key_version: "encryptionkey@0"
I20260805 22:42:06.858608 24504 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260805 22:42:06.859572 24521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:06.859840 24504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:06.859944 24504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "de7f3010b9274821a34c04af984b9e87"
format_stamp: "Formatted at 2026-08-05 22:42:06 on dist-test-slave-g170"
server_key: "1b90b452c48cdb8371a32a8fe1bd12c7"
server_key_iv: "5d7458d1f5b9a96a9d4e990f20f2d136"
server_key_version: "encryptionkey@0"
I20260805 22:42:06.860020 24504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:06.876338 24504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:06.878664 24504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:06.878800 24504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:06.879117 24504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:06.879539 24504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:06.879612 24504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:06.879637 24504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:06.879650 24504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:06.887622 24504 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:36297
I20260805 22:42:06.887732 24685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:36297 every 8 connection(s)
I20260805 22:42:06.888127 24504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:06.890933 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24504
I20260805 22:42:06.891057 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:06.891211 24341 external_mini_cluster.cc:1428] Setting key 31ba9e78eea6f1a95b8900a5cb9738ed
Aug 05 22:42:06 dist-test-slave-g170 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969726, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:06.900745 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:06.889295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:37353 (local address 127.23.197.126:33215)
0805 22:42:06.889349 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:06.889351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:06.889901 (+ 550us) server_negotiation.cc:407] Connection header received
0805 22:42:06.891490 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:06.891491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:06.891505 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:06.891558 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:06.892629 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:06.893452 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:06.894699 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:06.894939 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:06.895037 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:06.897678 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:06.897679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:06.897680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:06.897687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:06.899271 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:06.899698 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:06.899700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:06.899701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:06.899770 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:06.900030 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:06.900031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:06.900032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:06.900154 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:06.900255 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:06.900552 (+ 297us) server_negotiation.cc:299] Negotiation successful
0805 22:42:06.900651 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:42:06.901067 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:06.889293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33215 (local address 127.23.197.65:37353)
0805 22:42:06.889684 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:06.889770 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:06.891375 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:06.891662 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:06.891665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:06.892147 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:06.892550 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:06.892556 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:06.893562 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:06.893563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:06.894594 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:06.894596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:06.895050 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:06.895051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:06.895250 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:06.895900 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:06.895909 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:06.897575 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:06.899362 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:06.899371 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:06.899374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:06.899603 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:06.899841 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:06.899843 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:06.899843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:06.899956 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:06.900243 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:06.900251 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:06.900392 (+ 141us) client_negotiation.cc:769] Sending connection context
0805 22:42:06.900506 (+ 114us) client_negotiation.cc:240] Negotiation successful
0805 22:42:06.900675 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":208,"threads_started":1}
I20260805 22:42:06.901934 24686 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33215
I20260805 22:42:06.902050 24686 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:06.902293 24686 heartbeater.cc:511] Master 127.23.197.126:33215 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
I20260805 22:42:06.903348 24415 ts_manager.cc:194] Registered new tserver with Master: de7f3010b9274821a34c04af984b9e87 (127.23.197.65:36297)
I20260805 22:42:06.904503 24415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:37353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:06.923832 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33215
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:07.046597 24695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:07.046851 24695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:07.046880 24695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:07.048151 24695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:07.048197 24695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:07.048226 24695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:07.050004 24695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33215
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:07.050287 24695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:07.050535 24695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:07.057090 24701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:07.057137 24705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:07.057248 24695 server_base.cc:1034] running on GCE node
W20260805 22:42:07.057137 24702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:07.057550 24695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:07.057833 24695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:07.059015 24695 hybrid_clock.cc:648] HybridClock initialized: now 1785969727058998 us; error 71 us; skew 500 ppm
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969727, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:07.062633 24695 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:07.063205 24695 webserver.cc:460] Webserver started at http://127.23.197.66:42947/ using document root <none> and password file <none>
I20260805 22:42:07.063453 24695 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:07.063522 24695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:07.063678 24695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:07.065019 24695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "a7d39090a34640f09bffc74e1b0c16bb"
format_stamp: "Formatted at 2026-08-05 22:42:07 on dist-test-slave-g170"
server_key: "e2a8f92368873cd62aa6826101ef24a2"
server_key_iv: "7c8c543f162bcb6c0a064fa73ef7008c"
server_key_version: "encryptionkey@0"
I20260805 22:42:07.065433 24695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "a7d39090a34640f09bffc74e1b0c16bb"
format_stamp: "Formatted at 2026-08-05 22:42:07 on dist-test-slave-g170"
server_key: "e2a8f92368873cd62aa6826101ef24a2"
server_key_iv: "7c8c543f162bcb6c0a064fa73ef7008c"
server_key_version: "encryptionkey@0"
I20260805 22:42:07.067098 24695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:42:07.068074 24717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:07.068526 24695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:42:07.068665 24695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "a7d39090a34640f09bffc74e1b0c16bb"
format_stamp: "Formatted at 2026-08-05 22:42:07 on dist-test-slave-g170"
server_key: "e2a8f92368873cd62aa6826101ef24a2"
server_key_iv: "7c8c543f162bcb6c0a064fa73ef7008c"
server_key_version: "encryptionkey@0"
I20260805 22:42:07.068745 24695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:07.108690 24695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:07.111146 24695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:07.111366 24695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:07.111759 24695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:07.112164 24695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:07.112226 24695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:07.112269 24695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:07.112294 24695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:07.121970 24695 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:37885
I20260805 22:42:07.122030 24899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:37885 every 8 connection(s)
I20260805 22:42:07.122390 24695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969727, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:07.131105 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24695
I20260805 22:42:07.131222 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:07.131417 24341 external_mini_cluster.cc:1428] Setting key c882d30942ad16fc008ca84b2bc50e88
I20260805 22:42:07.132148 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:07.123199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:38533 (local address 127.23.197.126:33215)
0805 22:42:07.123283 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:07.123285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:07.123459 (+ 174us) server_negotiation.cc:407] Connection header received
0805 22:42:07.124004 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:07.124005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:07.124017 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:07.124075 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:07.125246 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.125779 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.126729 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.126907 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.126957 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:07.129109 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:07.129110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:07.129110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:07.129117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:07.130633 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:07.131042 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:07.131043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:07.131044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:07.131114 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:07.131414 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:07.131415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:07.131415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:07.131558 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:07.131634 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:07.131942 (+ 308us) server_negotiation.cc:299] Negotiation successful
0805 22:42:07.132028 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 22:42:07.132301 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:07.123147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33215 (local address 127.23.197.66:38533)
0805 22:42:07.123334 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:07.123371 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:07.123906 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:07.124130 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:07.124134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:07.124959 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:07.125162 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:07.125167 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.125869 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.125870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:07.126656 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:07.126658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.126978 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.126978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:07.127081 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:07.127544 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:07.127552 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:07.129026 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:07.130733 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:07.130740 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:07.130741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:07.130960 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:07.131223 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:07.131224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:07.131225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:07.131360 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:07.131629 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:07.131636 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:07.131815 (+ 179us) client_negotiation.cc:769] Sending connection context
0805 22:42:07.131904 (+ 89us) client_negotiation.cc:240] Negotiation successful
0805 22:42:07.132057 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":120,"threads_started":1}
I20260805 22:42:07.132886 24900 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33215
I20260805 22:42:07.132993 24900 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:07.133201 24900 heartbeater.cc:511] Master 127.23.197.126:33215 requested a full tablet report, sending...
I20260805 22:42:07.133613 24415 ts_manager.cc:194] Registered new tserver with Master: a7d39090a34640f09bffc74e1b0c16bb (127.23.197.66:37885)
I20260805 22:42:07.133994 24415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:38533
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:07.160552 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33215
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:07.248116 24907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:07.248313 24907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:07.248337 24907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:07.249449 24907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:07.249485 24907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:07.249511 24907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:07.250906 24907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33215
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:07.251093 24907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:07.251271 24907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:07.256435 24913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:07.256444 24917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:07.256451 24915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:07.256583 24907 server_base.cc:1034] running on GCE node
I20260805 22:42:07.256904 24907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:07.257118 24907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:07.258271 24907 hybrid_clock.cc:648] HybridClock initialized: now 1785969727258251 us; error 45 us; skew 500 ppm
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969727, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:07.260854 24907 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:07.261327 24907 webserver.cc:460] Webserver started at http://127.23.197.67:45883/ using document root <none> and password file <none>
I20260805 22:42:07.261588 24907 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:07.261664 24907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:07.261823 24907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:07.262884 24907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "df5119e60187410b975158397e963d53"
format_stamp: "Formatted at 2026-08-05 22:42:07 on dist-test-slave-g170"
server_key: "7897b320871baa10fa2a65bc94d6aee9"
server_key_iv: "2218f18ee474938fd99d16c8575c52df"
server_key_version: "encryptionkey@0"
I20260805 22:42:07.263270 24907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "df5119e60187410b975158397e963d53"
format_stamp: "Formatted at 2026-08-05 22:42:07 on dist-test-slave-g170"
server_key: "7897b320871baa10fa2a65bc94d6aee9"
server_key_iv: "2218f18ee474938fd99d16c8575c52df"
server_key_version: "encryptionkey@0"
I20260805 22:42:07.264842 24907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260805 22:42:07.265815 24925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:07.266081 24907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:07.266209 24907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "df5119e60187410b975158397e963d53"
format_stamp: "Formatted at 2026-08-05 22:42:07 on dist-test-slave-g170"
server_key: "7897b320871baa10fa2a65bc94d6aee9"
server_key_iv: "2218f18ee474938fd99d16c8575c52df"
server_key_version: "encryptionkey@0"
I20260805 22:42:07.266294 24907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:07.278899 24907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:07.281518 24907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:07.281744 24907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:07.282028 24907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:07.282372 24907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:07.282428 24907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:07.282465 24907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:07.282512 24907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:07.289438 24907 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:41077
I20260805 22:42:07.289492 25081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:41077 every 8 connection(s)
I20260805 22:42:07.289842 24907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:07.296499 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24907
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969727, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:07.296609 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:07.296815 24341 external_mini_cluster.cc:1428] Setting key 52bd990aad31803ad0004f96befc84c3
I20260805 22:42:07.300221 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:07.290637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:55497 (local address 127.23.197.126:33215)
0805 22:42:07.290761 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:07.290763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:07.290951 (+ 188us) server_negotiation.cc:407] Connection header received
0805 22:42:07.291554 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:07.291555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:07.291569 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:07.291636 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:07.292668 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.293301 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.294382 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.294600 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.294700 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:07.297274 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:07.297276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:07.297276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:07.297283 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:07.298792 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:07.299181 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:07.299182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:07.299182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:07.299241 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:07.299507 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:07.299508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:07.299508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:07.299638 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:07.299749 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:07.300072 (+ 323us) server_negotiation.cc:299] Negotiation successful
0805 22:42:07.300148 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 22:42:07.300413 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:07.290602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33215 (local address 127.23.197.67:55497)
0805 22:42:07.290809 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:07.290860 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:07.291434 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:07.291716 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:07.291722 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:07.292330 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:07.292548 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:07.292552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.293419 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.293420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:07.294265 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:07.294267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.294705 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.294706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:07.294843 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:07.295457 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:07.295466 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:07.297141 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:07.298866 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:07.298873 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:07.298875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:07.299108 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:07.299307 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:07.299308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:07.299308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:07.299415 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:07.299790 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:07.299798 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:07.299932 (+ 134us) client_negotiation.cc:769] Sending connection context
0805 22:42:07.300033 (+ 101us) client_negotiation.cc:240] Negotiation successful
0805 22:42:07.300180 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
I20260805 22:42:07.300987 25082 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33215
I20260805 22:42:07.301092 25082 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:07.301296 25082 heartbeater.cc:511] Master 127.23.197.126:33215 requested a full tablet report, sending...
I20260805 22:42:07.301748 24415 ts_manager.cc:194] Registered new tserver with Master: df5119e60187410b975158397e963d53 (127.23.197.67:41077)
I20260805 22:42:07.302131 24415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:55497
I20260805 22:42:07.309319 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969727, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969727, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:07.335052 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:07.326923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38078 (local address 127.23.197.65:36297)
0805 22:42:07.327193 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:07.327198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:07.327212 (+ 14us) server_negotiation.cc:407] Connection header received
0805 22:42:07.327240 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:07.327243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:07.327257 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:07.327337 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:07.328126 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.328928 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.329696 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.330043 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.330132 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:07.332400 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:07.332408 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:07.332409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:07.332416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:07.333991 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:07.334320 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:07.334323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:07.334324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:07.334380 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:07.334553 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:07.334554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:07.334554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:07.334682 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:07.334847 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:07.334905 (+ 58us) server_negotiation.cc:299] Negotiation successful
0805 22:42:07.334971 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":119,"threads_started":1}
W20260805 22:42:07.335521 24559 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38078
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:07.360550 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:07.353095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.23.197.65:36297)
0805 22:42:07.353182 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:07.353184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:07.353271 (+ 87us) server_negotiation.cc:407] Connection header received
0805 22:42:07.353385 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:07.353387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:07.353402 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:07.353471 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:07.354138 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.354635 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.355384 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:07.355609 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:07.355689 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:07.358189 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:07.358191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:07.358191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:07.358198 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:07.359622 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:07.359942 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:07.359943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:07.359943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:07.359996 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:07.360169 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:07.360169 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:07.360169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:07.360262 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:07.360348 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:07.360426 (+ 78us) server_negotiation.cc:299] Negotiation successful
0805 22:42:07.360485 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 22:42:07.361219 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24504
I20260805 22:42:07.374315 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24695
I20260805 22:42:07.384740 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24907
I20260805 22:42:07.398155 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24369
2026-08-05T22:42:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2317 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[25100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[25100](info): set up 2 sockets
Aug 05 22:42:07 dist-test-slave-g170 krb5kdc[25100](info): commencing operation
krb5kdc: starting...
W20260805 22:42:08.649289 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:08 dist-test-slave-g170 krb5kdc[25100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:08.775951 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33927
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:33927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:08.861083 25119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:08.861332 25119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:08.861369 25119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:08.862516 25119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:08.862583 25119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:08.862608 25119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:08.862628 25119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:08.862684 25119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:08.864221 25119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:33927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:08.864516 25119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:08.864755 25119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:08.870256 25133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:08.870203 25129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:08.870297 25119 server_base.cc:1034] running on GCE node
W20260805 22:42:08.870235 25130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:08.870779 25119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:08.871104 25119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:08.872299 25119 hybrid_clock.cc:648] HybridClock initialized: now 1785969728872284 us; error 47 us; skew 500 ppm
Aug 05 22:42:08 dist-test-slave-g170 krb5kdc[25100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969728, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:08.878402 25119 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:08.878880 25119 webserver.cc:460] Webserver started at http://127.23.197.126:43961/ using document root <none> and password file <none>
I20260805 22:42:08.879155 25119 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:08.879207 25119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:08.879347 25119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:08.880494 25119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "2f091591e2354f4590af33a255432ad5"
format_stamp: "Formatted at 2026-08-05 22:42:08 on dist-test-slave-g170"
server_key: "81632b7f40da121efc4364d278f08c1f"
server_key_iv: "df67ecbd9dcf0d96dc0928ccea07c22a"
server_key_version: "encryptionkey@0"
I20260805 22:42:08.880915 25119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "2f091591e2354f4590af33a255432ad5"
format_stamp: "Formatted at 2026-08-05 22:42:08 on dist-test-slave-g170"
server_key: "81632b7f40da121efc4364d278f08c1f"
server_key_iv: "df67ecbd9dcf0d96dc0928ccea07c22a"
server_key_version: "encryptionkey@0"
I20260805 22:42:08.882392 25119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:08.883354 25145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:08.883664 25119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:08.883783 25119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "2f091591e2354f4590af33a255432ad5"
format_stamp: "Formatted at 2026-08-05 22:42:08 on dist-test-slave-g170"
server_key: "81632b7f40da121efc4364d278f08c1f"
server_key_iv: "df67ecbd9dcf0d96dc0928ccea07c22a"
server_key_version: "encryptionkey@0"
I20260805 22:42:08.883862 25119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:08.898301 25119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:08.900635 25119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:08.900846 25119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:08.905328 25119 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:33927
I20260805 22:42:08.905362 25222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:33927 every 8 connection(s)
I20260805 22:42:08.905736 25119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:08.906737 25223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:08.909386 25223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5: Bootstrap starting.
I20260805 22:42:08.909994 25223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:08.910267 25223 log.cc:824] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:08.910971 25223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5: No bootstrap required, opened a new log
I20260805 22:42:08.911652 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 25119
I20260805 22:42:08.911819 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:08.912004 24341 external_mini_cluster.cc:1428] Setting key ab4901556af03834d6694ef852daa635
I20260805 22:42:08.913265 25223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f091591e2354f4590af33a255432ad5" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33927 } }
I20260805 22:42:08.913475 25223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:08.913534 25223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f091591e2354f4590af33a255432ad5, State: Initialized, Role: FOLLOWER
I20260805 22:42:08.913717 25223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f091591e2354f4590af33a255432ad5" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33927 } }
I20260805 22:42:08.913791 25223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:08.913841 25223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:08.913893 25223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:08.915373 25223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f091591e2354f4590af33a255432ad5" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33927 } }
I20260805 22:42:08.915503 25223 leader_election.cc:302] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f091591e2354f4590af33a255432ad5; no voters:
I20260805 22:42:08.915679 25223 leader_election.cc:288] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:08.915759 25229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:08.915964 25229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f091591e2354f4590af33a255432ad5, State: Running, Role: LEADER
I20260805 22:42:08.915982 25223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:08.916102 25229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f091591e2354f4590af33a255432ad5" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33927 } }
I20260805 22:42:08.916571 25230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f091591e2354f4590af33a255432ad5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f091591e2354f4590af33a255432ad5" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33927 } } }
I20260805 22:42:08.916581 25233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f091591e2354f4590af33a255432ad5. Latest consensus state: current_term: 1 leader_uuid: "2f091591e2354f4590af33a255432ad5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f091591e2354f4590af33a255432ad5" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33927 } } }
I20260805 22:42:08.916663 25230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:08.916666 25233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:08.916982 25238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:08.917584 25238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:08.919667 25238 catalog_manager.cc:1347] Generated new cluster ID: 50c443c2998b4c93b18af888bf1d4cd7
I20260805 22:42:08.919734 25238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:42:08 dist-test-slave-g170 krb5kdc[25100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:08.925917 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:08.912576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.23.197.126:33927)
0805 22:42:08.912840 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:08.912852 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:08.912894 (+ 42us) server_negotiation.cc:407] Connection header received
0805 22:42:08.913201 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:08.913210 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:08.913289 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:08.913474 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:08.914336 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:08.916093 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:08.917464 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:08.918220 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:08.918780 (+ 560us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:08.921409 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:08.921413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:08.921413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:08.921426 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:08.923518 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:08.923962 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:08.923965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:08.923966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:08.924097 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:08.924328 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:08.924328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:08.924329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:08.924671 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:08.925275 (+ 604us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:08.925351 (+ 76us) server_negotiation.cc:299] Negotiation successful
0805 22:42:08.925505 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":112,"threads_started":1}
I20260805 22:42:08.952193 25238 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:08.952854 25238 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:08.955099 25238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f091591e2354f4590af33a255432ad5: Generated new TSK 0
I20260805 22:42:08.955348 25238 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:08.987944 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33927
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:09.076084 25261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:09.076290 25261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:09.076314 25261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:09.077517 25261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:09.077560 25261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:09.077586 25261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:09.079077 25261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33927
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:09.079329 25261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:09.079569 25261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:09.084786 25270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:09.084833 25271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:09.084787 25275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:09.085096 25261 server_base.cc:1034] running on GCE node
I20260805 22:42:09.085332 25261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:09.085603 25261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:09.086779 25261 hybrid_clock.cc:648] HybridClock initialized: now 1785969729086753 us; error 42 us; skew 500 ppm
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:09.089533 25261 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:09.089979 25261 webserver.cc:460] Webserver started at http://127.23.197.65:37875/ using document root <none> and password file <none>
I20260805 22:42:09.090183 25261 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:09.090229 25261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:09.090330 25261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:09.091604 25261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "eff2dcd4957a420e9a71fce8b3cca5f4"
format_stamp: "Formatted at 2026-08-05 22:42:09 on dist-test-slave-g170"
server_key: "66cd169974b78d4b321b08615d4b31e0"
server_key_iv: "f69e189c4b20766df4fe26b8774f331f"
server_key_version: "encryptionkey@0"
I20260805 22:42:09.091997 25261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "eff2dcd4957a420e9a71fce8b3cca5f4"
format_stamp: "Formatted at 2026-08-05 22:42:09 on dist-test-slave-g170"
server_key: "66cd169974b78d4b321b08615d4b31e0"
server_key_iv: "f69e189c4b20766df4fe26b8774f331f"
server_key_version: "encryptionkey@0"
I20260805 22:42:09.093650 25261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:09.094502 25284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:09.094789 25261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:09.094973 25261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "eff2dcd4957a420e9a71fce8b3cca5f4"
format_stamp: "Formatted at 2026-08-05 22:42:09 on dist-test-slave-g170"
server_key: "66cd169974b78d4b321b08615d4b31e0"
server_key_iv: "f69e189c4b20766df4fe26b8774f331f"
server_key_version: "encryptionkey@0"
I20260805 22:42:09.095129 25261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:09.104461 25261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:09.106372 25261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:09.106539 25261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:09.106786 25261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:09.107100 25261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:09.107157 25261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:09.107203 25261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:09.107228 25261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:09.113121 25261 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:39847
I20260805 22:42:09.113160 25455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:39847 every 8 connection(s)
I20260805 22:42:09.113525 25261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:09.113894 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 25261
I20260805 22:42:09.114015 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:09.114207 24341 external_mini_cluster.cc:1428] Setting key 4ce73cb35e9da7611831224b77611bca
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
I20260805 22:42:09.126046 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.114412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:57027 (local address 127.23.197.126:33927)
0805 22:42:09.114479 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.114482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.114709 (+ 227us) server_negotiation.cc:407] Connection header received
0805 22:42:09.116056 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.116057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.116070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.116122 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:09.117067 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.117702 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.119425 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.119627 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.119685 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.122685 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:09.122687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:09.122687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:09.122698 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:09.124574 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.125033 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.125034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.125035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.125097 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.125349 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.125350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.125350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.125504 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:09.125582 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.125883 (+ 301us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.125978 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260805 22:42:09.126201 25461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.114316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33927 (local address 127.23.197.65:57027)
0805 22:42:09.114562 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:09.114611 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:09.115945 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:09.116194 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:09.116199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:09.116620 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:09.116943 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.116951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.117794 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.117795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.119332 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.119336 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.119724 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.119725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.119956 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.120601 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:09.120613 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:09.122588 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:09.124661 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:09.124667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:09.124669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:09.124954 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:09.125170 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:09.125170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:09.125171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:09.125274 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:09.125584 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:09.125590 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:09.125729 (+ 139us) client_negotiation.cc:769] Sending connection context
0805 22:42:09.125809 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 22:42:09.125930 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
I20260805 22:42:09.126849 25456 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33927
I20260805 22:42:09.127002 25456 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:09.127214 25456 heartbeater.cc:511] Master 127.23.197.126:33927 requested a full tablet report, sending...
I20260805 22:42:09.127939 25169 ts_manager.cc:194] Registered new tserver with Master: eff2dcd4957a420e9a71fce8b3cca5f4 (127.23.197.65:39847)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
I20260805 22:42:09.128906 25169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:57027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:09.144003 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33927
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:09.228586 25467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:09.228832 25467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:09.228858 25467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:09.230077 25467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:09.230118 25467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:09.230144 25467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:09.231576 25467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33927
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:09.231778 25467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:09.231966 25467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:09.237093 25475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:09.237098 25477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:09.237114 25467 server_base.cc:1034] running on GCE node
W20260805 22:42:09.237107 25474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:09.237526 25467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:09.237795 25467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:09.238973 25467 hybrid_clock.cc:648] HybridClock initialized: now 1785969729239021 us; error 92 us; skew 500 ppm
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:09.241655 25467 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:09.242188 25467 webserver.cc:460] Webserver started at http://127.23.197.66:34563/ using document root <none> and password file <none>
I20260805 22:42:09.242442 25467 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:09.242515 25467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:09.242668 25467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:09.243711 25467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "799cc483580d4a0b903362d748875e3c"
format_stamp: "Formatted at 2026-08-05 22:42:09 on dist-test-slave-g170"
server_key: "8bd30219d742e0790ae902472a6d16e8"
server_key_iv: "a01b1f3a6c58f09250974625550869dd"
server_key_version: "encryptionkey@0"
I20260805 22:42:09.244091 25467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "799cc483580d4a0b903362d748875e3c"
format_stamp: "Formatted at 2026-08-05 22:42:09 on dist-test-slave-g170"
server_key: "8bd30219d742e0790ae902472a6d16e8"
server_key_iv: "a01b1f3a6c58f09250974625550869dd"
server_key_version: "encryptionkey@0"
I20260805 22:42:09.245514 25467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260805 22:42:09.246287 25489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:09.246490 25467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:09.246605 25467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "799cc483580d4a0b903362d748875e3c"
format_stamp: "Formatted at 2026-08-05 22:42:09 on dist-test-slave-g170"
server_key: "8bd30219d742e0790ae902472a6d16e8"
server_key_iv: "a01b1f3a6c58f09250974625550869dd"
server_key_version: "encryptionkey@0"
I20260805 22:42:09.246681 25467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:09.269179 25467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:09.271257 25467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:09.271438 25467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:09.271688 25467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:09.272034 25467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:09.272092 25467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:09.272128 25467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:09.272157 25467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:09.279247 25467 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:46791
I20260805 22:42:09.279330 25660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:46791 every 8 connection(s)
I20260805 22:42:09.279686 25467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:09.280063 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 25467
I20260805 22:42:09.280165 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:09.280349 24341 external_mini_cluster.cc:1428] Setting key a1f92833fd68ca5320c3286d00473cc2
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
I20260805 22:42:09.291121 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.280445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:42829 (local address 127.23.197.126:33927)
0805 22:42:09.280498 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.280500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.280743 (+ 243us) server_negotiation.cc:407] Connection header received
0805 22:42:09.281873 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.281874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.281887 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.281945 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:09.282609 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.283459 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.284522 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.284863 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.284935 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.287762 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:09.287763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:09.287764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:09.287772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:09.289488 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.289926 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.289928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.289929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.289991 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.290266 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.290267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.290268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.290414 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:09.290483 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.290895 (+ 412us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.290992 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:42:09.291802 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.280443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33927 (local address 127.23.197.66:42829)
0805 22:42:09.280613 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:09.280656 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:09.281736 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:09.282021 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:09.282024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:09.282308 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:09.282508 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.282513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.283576 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.283577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.284430 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.284433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.284964 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.284965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.285090 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.285722 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:09.285732 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:09.287587 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:09.289590 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:09.289596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:09.289598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:09.289843 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:09.290069 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:09.290069 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:09.290070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:09.290175 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:09.290516 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:09.290524 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:09.290716 (+ 192us) client_negotiation.cc:769] Sending connection context
0805 22:42:09.291281 (+ 565us) client_negotiation.cc:240] Negotiation successful
0805 22:42:09.291486 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260805 22:42:09.292297 25662 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33927
I20260805 22:42:09.292405 25662 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:09.292641 25662 heartbeater.cc:511] Master 127.23.197.126:33927 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
I20260805 22:42:09.294713 25169 ts_manager.cc:194] Registered new tserver with Master: 799cc483580d4a0b903362d748875e3c (127.23.197.66:46791)
I20260805 22:42:09.295281 25169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:42829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:09.312706 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33927
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:09.402693 25667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:09.402905 25667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:09.402932 25667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:09.404124 25667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:09.404165 25667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:09.404193 25667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:09.405826 25667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33927
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:09.406081 25667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:09.406343 25667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:09.411444 25675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:09.411495 25679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:09.411444 25674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:09.411917 25667 server_base.cc:1034] running on GCE node
I20260805 22:42:09.412151 25667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:09.412393 25667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:09.413563 25667 hybrid_clock.cc:648] HybridClock initialized: now 1785969729413545 us; error 29 us; skew 500 ppm
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:09.416250 25667 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:09.416939 25667 webserver.cc:460] Webserver started at http://127.23.197.67:38995/ using document root <none> and password file <none>
I20260805 22:42:09.417230 25667 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:09.417306 25667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:09.417455 25667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:09.418668 25667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "0b67781627b04b40a853dace5f1cdbd8"
format_stamp: "Formatted at 2026-08-05 22:42:09 on dist-test-slave-g170"
server_key: "5b71c48e347d756f46a1ed19cac9655f"
server_key_iv: "19e928e52a629e76e8fa5ea3d0638efb"
server_key_version: "encryptionkey@0"
I20260805 22:42:09.419068 25667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "0b67781627b04b40a853dace5f1cdbd8"
format_stamp: "Formatted at 2026-08-05 22:42:09 on dist-test-slave-g170"
server_key: "5b71c48e347d756f46a1ed19cac9655f"
server_key_iv: "19e928e52a629e76e8fa5ea3d0638efb"
server_key_version: "encryptionkey@0"
I20260805 22:42:09.420542 25667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:09.421433 25692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:09.421628 25667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:42:09.421738 25667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "0b67781627b04b40a853dace5f1cdbd8"
format_stamp: "Formatted at 2026-08-05 22:42:09 on dist-test-slave-g170"
server_key: "5b71c48e347d756f46a1ed19cac9655f"
server_key_iv: "19e928e52a629e76e8fa5ea3d0638efb"
server_key_version: "encryptionkey@0"
I20260805 22:42:09.421829 25667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:09.439612 25667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:09.442353 25667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:09.442543 25667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:09.442857 25667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:09.443199 25667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:09.443261 25667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:09.443296 25667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:09.443338 25667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:09.449815 25667 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:38195
I20260805 22:42:09.449944 25855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:38195 every 8 connection(s)
I20260805 22:42:09.450264 25667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:09.459193 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 25667
I20260805 22:42:09.459321 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:09.459507 24341 external_mini_cluster.cc:1428] Setting key 715beea41e575f456c8bc733e0e34f75
I20260805 22:42:09.460562 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.451165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:57831 (local address 127.23.197.126:33927)
0805 22:42:09.451263 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.451265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.451372 (+ 107us) server_negotiation.cc:407] Connection header received
0805 22:42:09.452383 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.452385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.452397 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.452456 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:09.453092 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.453592 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.454661 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.454887 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.454997 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.457391 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:09.457393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:09.457393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:09.457400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:09.458746 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.459424 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.459425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.459425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.459506 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.459836 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.459837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.459837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.459954 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:09.460032 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.460362 (+ 330us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.460447 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 22:42:09.460784 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.451060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33927 (local address 127.23.197.67:57831)
0805 22:42:09.451231 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:09.451279 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:09.452262 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:09.452514 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:09.452518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:09.452813 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:09.453010 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.453015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.453729 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.453730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.454572 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.454574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.455012 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.455013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.455170 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.455770 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:09.455779 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:09.457284 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:09.458840 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:09.458847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:09.458849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:09.459304 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:09.459622 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:09.459623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:09.459623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:09.459755 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:09.460004 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:09.460011 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:09.460200 (+ 189us) client_negotiation.cc:769] Sending connection context
0805 22:42:09.460322 (+ 122us) client_negotiation.cc:240] Negotiation successful
0805 22:42:09.460484 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260805 22:42:09.461316 25856 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33927
I20260805 22:42:09.461421 25856 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:09.461602 25856 heartbeater.cc:511] Master 127.23.197.126:33927 requested a full tablet report, sending...
I20260805 22:42:09.461988 25169 ts_manager.cc:194] Registered new tserver with Master: 0b67781627b04b40a853dace5f1cdbd8 (127.23.197.67:38195)
I20260805 22:42:09.462327 25169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:57831
I20260805 22:42:09.471913 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:42:09.480290 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.473969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.23.197.126:33927)
0805 22:42:09.474050 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.474053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.474145 (+ 92us) server_negotiation.cc:407] Connection header received
0805 22:42:09.474242 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.474243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.474253 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.474318 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:09.474930 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.475429 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.476159 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.476360 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.476440 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.477808 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:09.477810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:09.477811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:09.477818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:09.479070 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.479437 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.479438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.479439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.479500 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.479707 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.479708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.479708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.479865 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:09.479978 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.480082 (+ 104us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.480168 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 22:42:09.483450 25169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:09.485488 25169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:09.493546 25876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.489233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.23.197.67:38195)
0805 22:42:09.489630 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.489636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.489647 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:09.489694 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.489696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.489714 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.489832 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:09.490039 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.490985 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.492364 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.493124 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.493215 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.493280 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.493411 (+ 131us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.493450 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":95,"threads_started":1}
I20260805 22:42:09.493536 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.489058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:38195 (local address 127.0.0.1:44222)
0805 22:42:09.489292 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:09.489338 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:09.489411 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:09.489839 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:09.489843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:09.489846 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:09.489939 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.489942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.491103 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.491104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.492268 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.492270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.493217 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.493218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.493301 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.493315 (+ 14us) client_negotiation.cc:769] Sending connection context
0805 22:42:09.493390 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 22:42:09.493448 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260805 22:42:09.493995 25872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.489070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:39847 (local address 127.0.0.1:54640)
0805 22:42:09.489436 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:09.489448 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:09.489530 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:09.490322 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:09.490324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:09.490326 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:09.490366 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.490368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.491656 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.491657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.492768 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.492772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.493767 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.493768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.493828 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.493832 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:09.493875 (+ 43us) client_negotiation.cc:240] Negotiation successful
0805 22:42:09.493914 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":1,"thread_start_us":129,"threads_started":1}
I20260805 22:42:09.494184 25878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.489303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:46791 (local address 127.0.0.1:34702)
0805 22:42:09.489525 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:09.489534 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:09.489575 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:09.489991 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:09.489992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:09.489995 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:09.490060 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.490062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.491638 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.491639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.492620 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.492622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.493961 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.493962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.494019 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.494023 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:09.494072 (+ 49us) client_negotiation.cc:240] Negotiation successful
0805 22:42:09.494112 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":12,"thread_start_us":72,"threads_started":1}
I20260805 22:42:09.494257 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.489215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54640 (local address 127.23.197.65:39847)
0805 22:42:09.490006 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.490011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.490021 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:09.490060 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.490062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.490080 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.490183 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:09.490455 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.491539 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.492866 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.493581 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.493656 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.493729 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.494119 (+ 390us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.494169 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":90,"threads_started":1}
I20260805 22:42:09.494544 25757 tablet_service.cc:1467] Processing CreateTablet for tablet a1a13bcfb3f64bac8c401e86b9c5d222 (DEFAULT_TABLE table=test-table [id=2a660b0567f34487baddc45edcad26bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:09.494889 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.489567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.23.197.66:46791)
0805 22:42:09.489790 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.489794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.489802 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:09.489837 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.489838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.489870 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.490222 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:09.490319 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.491487 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.492715 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.493499 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.494671 (+ 1172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.494740 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.494766 (+ 26us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.494810 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":115,"threads_started":1}
I20260805 22:42:09.494980 25757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a13bcfb3f64bac8c401e86b9c5d222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:09.495010 25359 tablet_service.cc:1467] Processing CreateTablet for tablet a1a13bcfb3f64bac8c401e86b9c5d222 (DEFAULT_TABLE table=test-table [id=2a660b0567f34487baddc45edcad26bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:09.495500 25359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a13bcfb3f64bac8c401e86b9c5d222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:09.495668 25560 tablet_service.cc:1467] Processing CreateTablet for tablet a1a13bcfb3f64bac8c401e86b9c5d222 (DEFAULT_TABLE table=test-table [id=2a660b0567f34487baddc45edcad26bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:09.495995 25560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a13bcfb3f64bac8c401e86b9c5d222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:09.498148 25881 tablet_bootstrap.cc:492] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8: Bootstrap starting.
I20260805 22:42:09.499132 25883 tablet_bootstrap.cc:492] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4: Bootstrap starting.
I20260805 22:42:09.499166 25881 tablet_bootstrap.cc:654] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:09.499595 25884 tablet_bootstrap.cc:492] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c: Bootstrap starting.
I20260805 22:42:09.499985 25883 tablet_bootstrap.cc:654] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:09.500095 25881 log.cc:824] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:09.500252 25884 tablet_bootstrap.cc:654] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:09.500569 25883 log.cc:824] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:09.500962 25884 log.cc:824] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:09.501130 25881 tablet_bootstrap.cc:492] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8: No bootstrap required, opened a new log
I20260805 22:42:09.501214 25881 ts_tablet_manager.cc:1397] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:42:09.501410 25883 tablet_bootstrap.cc:492] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4: No bootstrap required, opened a new log
I20260805 22:42:09.501485 25883 ts_tablet_manager.cc:1397] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260805 22:42:09.501995 25884 tablet_bootstrap.cc:492] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c: No bootstrap required, opened a new log
I20260805 22:42:09.502056 25884 ts_tablet_manager.cc:1397] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 22:42:09.502846 25883 raft_consensus.cc:348] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } }
I20260805 22:42:09.502983 25883 raft_consensus.cc:374] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:09.503005 25883 raft_consensus.cc:729] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff2dcd4957a420e9a71fce8b3cca5f4, State: Initialized, Role: FOLLOWER
I20260805 22:42:09.503130 25883 consensus_queue.cc:260] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } }
I20260805 22:42:09.503443 25883 ts_tablet_manager.cc:1428] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260805 22:42:09.503372 25881 raft_consensus.cc:348] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } }
I20260805 22:42:09.503540 25881 raft_consensus.cc:374] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:09.503574 25881 raft_consensus.cc:729] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b67781627b04b40a853dace5f1cdbd8, State: Initialized, Role: FOLLOWER
I20260805 22:42:09.503629 25456 heartbeater.cc:503] Master 127.23.197.126:33927 was elected leader, sending a full tablet report...
I20260805 22:42:09.503700 25881 consensus_queue.cc:260] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } }
I20260805 22:42:09.504043 25856 heartbeater.cc:503] Master 127.23.197.126:33927 was elected leader, sending a full tablet report...
I20260805 22:42:09.504174 25884 raft_consensus.cc:348] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } }
I20260805 22:42:09.504374 25884 raft_consensus.cc:374] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:09.504431 25884 raft_consensus.cc:729] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799cc483580d4a0b903362d748875e3c, State: Initialized, Role: FOLLOWER
I20260805 22:42:09.504583 25884 consensus_queue.cc:260] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } }
I20260805 22:42:09.504662 25881 ts_tablet_manager.cc:1428] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:42:09.504922 25884 ts_tablet_manager.cc:1428] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260805 22:42:09.504926 25662 heartbeater.cc:503] Master 127.23.197.126:33927 was elected leader, sending a full tablet report...
W20260805 22:42:09.530319 25663 tablet.cc:2369] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:42:09.571254 25892 raft_consensus.cc:482] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:09.571385 25892 raft_consensus.cc:504] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } }
I20260805 22:42:09.571956 25892 leader_election.cc:288] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eff2dcd4957a420e9a71fce8b3cca5f4 (127.23.197.65:39847), 799cc483580d4a0b903362d748875e3c (127.23.197.66:46791)
I20260805 22:42:09.574931 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.572148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:39847 (local address 127.23.197.67:43101)
0805 22:42:09.572198 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:09.572212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:09.572288 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:09.572479 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:09.572480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:09.572482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:09.572558 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.572560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.573288 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.573289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.574089 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.574092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.574657 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.574658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.574746 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.574749 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:09.574818 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 22:42:09.574859 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":1}
I20260805 22:42:09.574931 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.572270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:43101 (local address 127.23.197.65:39847)
0805 22:42:09.572319 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.572322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.572346 (+ 24us) server_negotiation.cc:407] Connection header received
0805 22:42:09.572380 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.572381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.572394 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.572470 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:09.572637 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.573183 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.574186 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.574568 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.574660 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.574670 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.574825 (+ 155us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.574865 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:42:09.575276 25894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.572150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:46791 (local address 127.23.197.67:60375)
0805 22:42:09.572450 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:09.572459 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:09.572528 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:09.572668 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:09.572669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:09.572671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:09.572746 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.572749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.573478 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.573479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.574262 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:09.574264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.574841 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.574842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:09.574917 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.574927 (+ 10us) client_negotiation.cc:769] Sending connection context
0805 22:42:09.575138 (+ 211us) client_negotiation.cc:240] Negotiation successful
0805 22:42:09.575179 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":60,"thread_start_us":123,"threads_started":1}
I20260805 22:42:09.575289 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.572269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:60375 (local address 127.23.197.66:46791)
0805 22:42:09.572330 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.572332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.572527 (+ 195us) server_negotiation.cc:407] Connection header received
0805 22:42:09.572579 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.572580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.572590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.572664 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:09.572842 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.573384 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.574360 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.574752 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.574839 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.574848 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.575178 (+ 330us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.575226 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 22:42:09.575317 25386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a13bcfb3f64bac8c401e86b9c5d222" candidate_uuid: "0b67781627b04b40a853dace5f1cdbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" is_pre_election: true
I20260805 22:42:09.575482 25386 raft_consensus.cc:2454] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b67781627b04b40a853dace5f1cdbd8 in term 0.
I20260805 22:42:09.575742 25694 leader_election.cc:302] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b67781627b04b40a853dace5f1cdbd8, eff2dcd4957a420e9a71fce8b3cca5f4; no voters:
I20260805 22:42:09.575791 25589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a13bcfb3f64bac8c401e86b9c5d222" candidate_uuid: "0b67781627b04b40a853dace5f1cdbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799cc483580d4a0b903362d748875e3c" is_pre_election: true
I20260805 22:42:09.575917 25892 raft_consensus.cc:2781] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:09.575949 25589 raft_consensus.cc:2454] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b67781627b04b40a853dace5f1cdbd8 in term 0.
I20260805 22:42:09.575971 25892 raft_consensus.cc:482] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:09.575989 25892 raft_consensus.cc:3037] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:09.576810 25892 raft_consensus.cc:504] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } }
I20260805 22:42:09.576957 25892 leader_election.cc:288] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [CANDIDATE]: Term 1 election: Requested vote from peers eff2dcd4957a420e9a71fce8b3cca5f4 (127.23.197.65:39847), 799cc483580d4a0b903362d748875e3c (127.23.197.66:46791)
I20260805 22:42:09.577164 25386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a13bcfb3f64bac8c401e86b9c5d222" candidate_uuid: "0b67781627b04b40a853dace5f1cdbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4"
I20260805 22:42:09.577183 25589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a13bcfb3f64bac8c401e86b9c5d222" candidate_uuid: "0b67781627b04b40a853dace5f1cdbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799cc483580d4a0b903362d748875e3c"
I20260805 22:42:09.577265 25386 raft_consensus.cc:3037] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:09.577262 25589 raft_consensus.cc:3037] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:09.577996 25589 raft_consensus.cc:2454] T a1a13bcfb3f64bac8c401e86b9c5d222 P 799cc483580d4a0b903362d748875e3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b67781627b04b40a853dace5f1cdbd8 in term 1.
I20260805 22:42:09.577998 25386 raft_consensus.cc:2454] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b67781627b04b40a853dace5f1cdbd8 in term 1.
I20260805 22:42:09.578183 25696 leader_election.cc:302] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b67781627b04b40a853dace5f1cdbd8, 799cc483580d4a0b903362d748875e3c; no voters:
I20260805 22:42:09.578300 25892 raft_consensus.cc:2781] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:09.578473 25892 raft_consensus.cc:686] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [term 1 LEADER]: Becoming Leader. State: Replica: 0b67781627b04b40a853dace5f1cdbd8, State: Running, Role: LEADER
I20260805 22:42:09.578598 25892 consensus_queue.cc:237] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } }
I20260805 22:42:09.579509 25166 catalog_manager.cc:5495] T a1a13bcfb3f64bac8c401e86b9c5d222 P 0b67781627b04b40a853dace5f1cdbd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b67781627b04b40a853dace5f1cdbd8 (127.23.197.67). New cstate: current_term: 1 leader_uuid: "0b67781627b04b40a853dace5f1cdbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b67781627b04b40a853dace5f1cdbd8" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eff2dcd4957a420e9a71fce8b3cca5f4" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "799cc483580d4a0b903362d748875e3c" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46791 } health_report { overall_health: UNKNOWN } } }
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:09.597087 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.588621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.23.197.65:39847)
0805 22:42:09.588675 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.588678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.588967 (+ 289us) server_negotiation.cc:407] Connection header received
0805 22:42:09.589190 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.589191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.589202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.589265 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:09.589905 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.590360 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.591128 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.591319 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.591378 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.594172 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:09.594181 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:09.594182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:09.594188 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:09.595948 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.596284 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.596287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.596288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.596345 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.596512 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.596513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.596513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.596673 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:09.596854 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.596896 (+ 42us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.596962 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260805 22:42:09.614324 25457 tablet.cc:2369] T a1a13bcfb3f64bac8c401e86b9c5d222 P eff2dcd4957a420e9a71fce8b3cca5f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:09.622996 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.615100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.23.197.65:39847)
0805 22:42:09.615168 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.615170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.615287 (+ 117us) server_negotiation.cc:407] Connection header received
0805 22:42:09.615384 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.615386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.615404 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.615473 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:09.616138 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.616734 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.617615 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.617820 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.617901 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.620454 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:09.620455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:09.620456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:09.620463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:09.621857 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.622226 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.622227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.622227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.622286 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.622507 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.622508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.622508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.622606 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:09.622692 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.622858 (+ 166us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.622927 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:09.648132 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:09.640210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.23.197.65:39847)
0805 22:42:09.640305 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:09.640308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:09.640408 (+ 100us) server_negotiation.cc:407] Connection header received
0805 22:42:09.640513 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:09.640514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:09.640525 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:09.640601 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:09.641364 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.641865 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.642626 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:09.642881 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:09.642967 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:09.645465 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:09.645467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:09.645467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:09.645475 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:09.647057 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.647428 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.647429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.647430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.647484 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:09.647680 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:09.647680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:09.647681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:09.647808 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:09.647886 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:09.648002 (+ 116us) server_negotiation.cc:299] Negotiation successful
0805 22:42:09.648061 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260805 22:42:09.648854 25329 tablet_service.cc:2908] Rejecting scan request for tablet a1a13bcfb3f64bac8c401e86b9c5d222: Uninitialized: clean time has not yet been initialized
I20260805 22:42:09.649946 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 25261
I20260805 22:42:09.669756 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 25467
I20260805 22:42:09.680539 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 25667
I20260805 22:42:09.694011 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 25119
2026-08-05T22:42:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2296 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25917](info): set up 2 sockets
Aug 05 22:42:09 dist-test-slave-g170 krb5kdc[25917](info): commencing operation
krb5kdc: starting...
W20260805 22:42:10.935261 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:11.063047 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41343
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:41343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:11.148439 25938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:11.148713 25938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:11.148758 25938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:11.149955 25938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:11.150025 25938 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260805 22:42:11.150050 25938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:11.150085 25938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:11.150133 25938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:11.150161 25938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:11.151767 25938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:41343
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:11.152051 25938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:11.152305 25938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:11.157657 25949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:11.157716 25945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:11.157657 25946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:11.157850 25938 server_base.cc:1034] running on GCE node
I20260805 22:42:11.158084 25938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:11.158427 25938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:11.159580 25938 hybrid_clock.cc:648] HybridClock initialized: now 1785969731159525 us; error 74 us; skew 500 ppm
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:11.162341 25938 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:11.162856 25938 webserver.cc:460] Webserver started at http://127.23.197.126:42697/ using document root <none> and password file <none>
I20260805 22:42:11.163116 25938 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:11.163189 25938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:11.163350 25938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:11.164465 25938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "6b36b33e447343a98db2b15903f38c9c"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "ca418294d27d1d8c63b631168c2cca8c"
server_key_iv: "4f53e750123c16098c1129fc58a7850d"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.164932 25938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "6b36b33e447343a98db2b15903f38c9c"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "ca418294d27d1d8c63b631168c2cca8c"
server_key_iv: "4f53e750123c16098c1129fc58a7850d"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.166396 25938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:11.167261 25958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:11.167518 25938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:11.167647 25938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "6b36b33e447343a98db2b15903f38c9c"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "ca418294d27d1d8c63b631168c2cca8c"
server_key_iv: "4f53e750123c16098c1129fc58a7850d"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.167732 25938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:11.187637 25938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:11.190287 25938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:11.190474 25938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:11.196089 25938 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:41343
I20260805 22:42:11.196158 26054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:41343 every 8 connection(s)
I20260805 22:42:11.196527 25938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:11.197256 26056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:11.198874 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 25938
I20260805 22:42:11.198994 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:11.199168 24341 external_mini_cluster.cc:1428] Setting key e06ba8bef85737a6499c1b3ca606e0a6
I20260805 22:42:11.199936 26056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c: Bootstrap starting.
I20260805 22:42:11.200552 26056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:11.200963 26056 log.cc:824] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:11.201789 26056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c: No bootstrap required, opened a new log
I20260805 22:42:11.204243 26056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b36b33e447343a98db2b15903f38c9c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41343 } }
I20260805 22:42:11.204463 26056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:11.204507 26056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b36b33e447343a98db2b15903f38c9c, State: Initialized, Role: FOLLOWER
I20260805 22:42:11.204643 26056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b36b33e447343a98db2b15903f38c9c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41343 } }
I20260805 22:42:11.204702 26056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:11.204723 26056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:11.204751 26056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:11.205560 26056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b36b33e447343a98db2b15903f38c9c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41343 } }
I20260805 22:42:11.205685 26056 leader_election.cc:302] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b36b33e447343a98db2b15903f38c9c; no voters:
I20260805 22:42:11.205871 26056 leader_election.cc:288] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:11.205936 26063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:11.206105 26063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [term 1 LEADER]: Becoming Leader. State: Replica: 6b36b33e447343a98db2b15903f38c9c, State: Running, Role: LEADER
I20260805 22:42:11.206164 26056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:11.206203 26063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b36b33e447343a98db2b15903f38c9c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41343 } }
I20260805 22:42:11.206537 26064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b36b33e447343a98db2b15903f38c9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b36b33e447343a98db2b15903f38c9c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41343 } } }
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:11.206527 26063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b36b33e447343a98db2b15903f38c9c. Latest consensus state: current_term: 1 leader_uuid: "6b36b33e447343a98db2b15903f38c9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b36b33e447343a98db2b15903f38c9c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41343 } } }
I20260805 22:42:11.206775 26063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:11.206821 26064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:11.207015 26079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:11.207886 26079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:11.209949 26079 catalog_manager.cc:1347] Generated new cluster ID: 02c71aa207734e4286f446b3512e344b
I20260805 22:42:11.210003 26079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:11.211020 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.199749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47616 (local address 127.23.197.126:41343)
0805 22:42:11.199927 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.199936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.199971 (+ 35us) server_negotiation.cc:407] Connection header received
0805 22:42:11.200454 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.200461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.200509 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.200639 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:11.201507 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.202654 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.203821 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.204225 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.204309 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.207322 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:11.207325 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:11.207326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:11.207339 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:11.209241 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.209702 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.209705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.209706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.209813 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.210061 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.210070 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.210070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.210368 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:11.210511 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.210676 (+ 165us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.210815 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260805 22:42:11.223492 26079 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:11.224108 26079 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:11.226720 26079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c: Generated new TSK 0
I20260805 22:42:11.226925 26079 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:11.258163 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41343
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:11.347582 26095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:11.347851 26095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:11.347896 26095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:11.349195 26095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:11.349261 26095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:11.349295 26095 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 22:42:11.349339 26095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:11.350859 26095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.197.126:41343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:11.351140 26095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:11.351411 26095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:11.356706 26103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:11.356715 26100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:11.356972 26095 server_base.cc:1034] running on GCE node
W20260805 22:42:11.356745 26106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:11.357277 26095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:11.357535 26095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:11.358812 26095 hybrid_clock.cc:648] HybridClock initialized: now 1785969731358791 us; error 44 us; skew 500 ppm
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:11.361398 26095 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:11.361802 26095 webserver.cc:460] Webserver started at http://127.23.197.65:35141/ using document root <none> and password file <none>
I20260805 22:42:11.362048 26095 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:11.362097 26095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:11.362274 26095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:11.363428 26095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "c347b293ab9f4e29807ccf325869ae96"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "253f2360989c0bb7793c083d20277130"
server_key_iv: "ac2d51dfc9621f864945fe05aed9ed33"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.363804 26095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "c347b293ab9f4e29807ccf325869ae96"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "253f2360989c0bb7793c083d20277130"
server_key_iv: "ac2d51dfc9621f864945fe05aed9ed33"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.365247 26095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:11.366086 26115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:11.366315 26095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:11.366402 26095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "c347b293ab9f4e29807ccf325869ae96"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "253f2360989c0bb7793c083d20277130"
server_key_iv: "ac2d51dfc9621f864945fe05aed9ed33"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.366457 26095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:11.385578 26095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:11.388517 26095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:11.388713 26095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:11.389431 26095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:11.389478 26095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:11.389501 26095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:11.389513 26095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:11.396041 26095 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:36539
I20260805 22:42:11.396144 26281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:36539 every 8 connection(s)
I20260805 22:42:11.396498 26095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:11.400511 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.390057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:33507 (local address 127.23.197.126:41343)
0805 22:42:11.390105 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.390107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.390360 (+ 253us) server_negotiation.cc:407] Connection header received
0805 22:42:11.391021 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.391022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.391035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.391106 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:11.391957 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.392853 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.394359 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.394737 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.394837 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.397620 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:11.397621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:11.397621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:11.397629 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:11.399114 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.399522 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.399524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.399524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.399586 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.399898 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.399899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.399899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.400020 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:11.400079 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.400354 (+ 275us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.400440 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:42:11.400609 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.390066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41343 (local address 127.23.197.65:33507)
0805 22:42:11.390244 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.390285 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.390917 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.391180 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.391185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.391578 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:11.391871 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.391877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.392973 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.392974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.394260 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.394263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.394851 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.394853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.395025 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.395601 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:11.395611 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:11.397509 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:11.399204 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.399211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.399213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.399443 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.399708 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.399709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.399710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.399832 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.400084 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:11.400090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:11.400209 (+ 119us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.400320 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.400447 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260805 22:42:11.401168 26282 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41343
I20260805 22:42:11.401291 26282 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:11.401499 26282 heartbeater.cc:511] Master 127.23.197.126:41343 requested a full tablet report, sending...
I20260805 22:42:11.402148 25979 ts_manager.cc:194] Registered new tserver with Master: c347b293ab9f4e29807ccf325869ae96 (127.23.197.65:36539)
I20260805 22:42:11.403164 25979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:33507
I20260805 22:42:11.404873 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 26095
I20260805 22:42:11.405007 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:11.405192 24341 external_mini_cluster.cc:1428] Setting key 0f15094ab2b6219d531622170a0d5b1a
I20260805 22:42:11.409487 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.403353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:42547 (local address 127.23.197.126:41343)
0805 22:42:11.403410 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.403412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.403419 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:42:11.403437 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.403438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.403447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.403511 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:11.403877 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.404354 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.405444 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.405653 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.405733 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.406782 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:11.406784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:11.406784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:11.406791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:11.408289 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.408648 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.408649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.408650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.408706 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.408945 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.408946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.408946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.409069 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:11.409142 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.409311 (+ 169us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.409391 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260805 22:42:11.409499 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.402291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41343 (local address 127.23.197.65:42547)
0805 22:42:11.402457 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.402467 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.402511 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.403531 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.403533 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.403709 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:11.403796 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.403798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.404451 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.404452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.405342 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.405345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.405755 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.405756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.405844 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.406297 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:11.406302 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:11.406695 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:11.408393 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.408394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.408395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.408572 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.408804 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.408805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.408806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.408869 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.409146 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:11.409147 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:11.409196 (+ 49us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.409281 (+ 85us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.409391 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:11.436018 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41343
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:11.524860 26295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:11.525054 26295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:11.525079 26295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:11.526232 26295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:11.526275 26295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:11.526296 26295 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 22:42:11.526358 26295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:11.527770 26295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.197.126:41343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:11.528000 26295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:11.528229 26295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:11.533334 26302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:11.533365 26303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:11.533533 26295 server_base.cc:1034] running on GCE node
W20260805 22:42:11.533318 26305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:11.533919 26295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:11.534173 26295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:11.535434 26295 hybrid_clock.cc:648] HybridClock initialized: now 1785969731535409 us; error 37 us; skew 500 ppm
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:11.538128 26295 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:11.538589 26295 webserver.cc:460] Webserver started at http://127.23.197.66:44169/ using document root <none> and password file <none>
I20260805 22:42:11.538865 26295 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:11.538940 26295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:11.539093 26295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:11.540177 26295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "9fc979ec85fc4bfe8eb205341672dea0"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "f6794baf7c2172dbe1b68892b34a19ee"
server_key_iv: "278a0a5bf8f6cd3cae1c6b5d40d2bd12"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.540552 26295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "9fc979ec85fc4bfe8eb205341672dea0"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "f6794baf7c2172dbe1b68892b34a19ee"
server_key_iv: "278a0a5bf8f6cd3cae1c6b5d40d2bd12"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.542059 26295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:11.542836 26315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:11.543011 26295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:11.543113 26295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "9fc979ec85fc4bfe8eb205341672dea0"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "f6794baf7c2172dbe1b68892b34a19ee"
server_key_iv: "278a0a5bf8f6cd3cae1c6b5d40d2bd12"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.543193 26295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:11.581266 26295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:11.583743 26295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:11.583861 26295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:11.584372 26295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:11.584443 26295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:11.584496 26295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:11.584527 26295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:11.592010 26295 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:44307
I20260805 22:42:11.592098 26479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:44307 every 8 connection(s)
I20260805 22:42:11.592453 26295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:11.594918 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.585037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:36393 (local address 127.23.197.126:41343)
0805 22:42:11.585093 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.585096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.585319 (+ 223us) server_negotiation.cc:407] Connection header received
0805 22:42:11.585908 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.585910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.585924 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.585990 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:11.586867 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.587702 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.589147 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.589507 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.589593 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.591870 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:11.591872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:11.591872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:11.591882 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:11.593488 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.593925 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.593927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.593927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.593994 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.594260 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.594261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.594261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.594383 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:11.594453 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.594730 (+ 277us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.594810 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260805 22:42:11.595141 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.584974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41343 (local address 127.23.197.66:36393)
0805 22:42:11.585182 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.585220 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.585786 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.586066 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.586071 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.586428 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:11.586729 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.586737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.587849 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.587850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.589076 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.589079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.589622 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.589624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.589754 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.590305 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:11.590314 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:11.591758 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:11.593596 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.593603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.593605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.593840 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.594069 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.594070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.594070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.594178 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.594457 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:11.594463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:11.594599 (+ 136us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.594708 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.594912 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260805 22:42:11.595638 26480 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41343
I20260805 22:42:11.595723 26480 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:11.595908 26480 heartbeater.cc:511] Master 127.23.197.126:41343 requested a full tablet report, sending...
I20260805 22:42:11.596292 25980 ts_manager.cc:194] Registered new tserver with Master: 9fc979ec85fc4bfe8eb205341672dea0 (127.23.197.66:44307)
I20260805 22:42:11.596688 25980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:36393
I20260805 22:42:11.602085 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.596830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:38109 (local address 127.23.197.126:41343)
0805 22:42:11.596869 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.596871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.597008 (+ 137us) server_negotiation.cc:407] Connection header received
0805 22:42:11.597057 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.597057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.597067 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.597119 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:11.597480 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.597986 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.598673 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.598894 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.598979 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.599744 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:11.599745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:11.599746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:11.599752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:11.601021 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.601378 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.601379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.601379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.601429 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.601586 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.601586 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.601586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.601753 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:11.601809 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.601959 (+ 150us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.602027 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:42:11.602099 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.596715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41343 (local address 127.23.197.66:38109)
0805 22:42:11.596934 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.596947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.597004 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.597141 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.597142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.597346 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:11.597406 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.597407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.598087 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.598088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.598590 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.598591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.598967 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.598967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.599021 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.599377 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:11.599381 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:11.599641 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:11.601100 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.601101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.601101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.601325 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.601486 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.601486 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.601487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.601540 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.601813 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:11.601814 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:11.601851 (+ 37us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.601940 (+ 89us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.602029 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":99,"threads_started":1}
I20260805 22:42:11.602516 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 26295
I20260805 22:42:11.602617 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:11.602830 24341 external_mini_cluster.cc:1428] Setting key dc536185560b58f1cb9ca2b8996033c4
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:11.631975 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41343
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:11.717482 26491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:11.717684 26491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:11.717707 26491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:11.718899 26491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:11.718950 26491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:11.718971 26491 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 22:42:11.718989 26491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:11.720420 26491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.197.126:41343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:11.720598 26491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:11.720834 26491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:11.725895 26500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:11.725901 26497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:11.725919 26496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:11.725919 26491 server_base.cc:1034] running on GCE node
I20260805 22:42:11.726336 26491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:11.726537 26491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:11.727726 26491 hybrid_clock.cc:648] HybridClock initialized: now 1785969731727710 us; error 33 us; skew 500 ppm
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:11.730276 26491 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:11.730752 26491 webserver.cc:460] Webserver started at http://127.23.197.67:37183/ using document root <none> and password file <none>
I20260805 22:42:11.730955 26491 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:11.730999 26491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:11.731091 26491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:11.732123 26491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "7fef6214b78d4b64975bbae620d69f75"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "a749127988ae63e0af5e62975b32f6ce"
server_key_iv: "3219b3b05fd82dc2554b8bb6ab8c911e"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.732440 26491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "7fef6214b78d4b64975bbae620d69f75"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "a749127988ae63e0af5e62975b32f6ce"
server_key_iv: "3219b3b05fd82dc2554b8bb6ab8c911e"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.733915 26491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:11.734774 26507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:11.735006 26491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:11.735103 26491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "7fef6214b78d4b64975bbae620d69f75"
format_stamp: "Formatted at 2026-08-05 22:42:11 on dist-test-slave-g170"
server_key: "a749127988ae63e0af5e62975b32f6ce"
server_key_iv: "3219b3b05fd82dc2554b8bb6ab8c911e"
server_key_version: "encryptionkey@0"
I20260805 22:42:11.735176 26491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:11.747745 26491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:11.750178 26491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:11.750298 26491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:11.750783 26491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:11.750820 26491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:11.750844 26491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:11.750860 26491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:11.758456 26491 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:42071
I20260805 22:42:11.758605 26675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:42071 every 8 connection(s)
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:11.758948 26491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:11.761953 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.751402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:37273 (local address 127.23.197.126:41343)
0805 22:42:11.751453 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.751455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.751718 (+ 263us) server_negotiation.cc:407] Connection header received
0805 22:42:11.752338 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.752340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.752357 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.752434 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:11.753462 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.754299 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.756018 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.756345 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.756413 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.759371 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:11.759373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:11.759374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:11.759382 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:11.760626 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.761119 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.761120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.761121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.761180 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.761422 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.761423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.761423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.761528 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:11.761577 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.761821 (+ 244us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.761888 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:42:11.762090 26523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.751407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41343 (local address 127.23.197.67:37273)
0805 22:42:11.751588 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.751625 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.752189 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.752532 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.752536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.753023 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:11.753345 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.753351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.754421 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.754423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.755911 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.755915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.756446 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.756448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.756604 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.757250 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:11.757259 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:11.759275 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:11.760805 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.760813 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.760816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.761047 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.761250 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.761250 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.761251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.761357 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.761581 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:11.761585 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:11.761702 (+ 117us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.761779 (+ 77us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.761898 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260805 22:42:11.762614 26677 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41343
I20260805 22:42:11.762713 26677 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:11.762908 26677 heartbeater.cc:511] Master 127.23.197.126:41343 requested a full tablet report, sending...
I20260805 22:42:11.763353 25980 ts_manager.cc:194] Registered new tserver with Master: 7fef6214b78d4b64975bbae620d69f75 (127.23.197.67:42071)
I20260805 22:42:11.763797 25980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:37273
I20260805 22:42:11.767599 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 26491
I20260805 22:42:11.767696 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:11.767889 24341 external_mini_cluster.cc:1428] Setting key 8d633853a28449ca857448bd7118dce4
I20260805 22:42:11.768924 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:42:11.770249 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.763802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41343 (local address 127.23.197.67:59855)
0805 22:42:11.763995 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.764007 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.764064 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.764221 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.764222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.764432 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:11.764532 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.764534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.765314 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.765314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.765848 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.765849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.766229 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.766230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.766283 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.766727 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:11.766731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:11.767012 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:11.769085 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.769086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.769086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.769289 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.769530 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:11.769531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:11.769531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:11.769598 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:11.769893 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:11.769894 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:11.769943 (+ 49us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.770030 (+ 87us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.770143 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":3,"thread_start_us":81,"threads_started":1}
I20260805 22:42:11.770233 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.763873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:59855 (local address 127.23.197.126:41343)
0805 22:42:11.763924 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.763926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.764077 (+ 151us) server_negotiation.cc:407] Connection header received
0805 22:42:11.764127 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.764128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.764142 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.764204 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:11.764672 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.765223 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.765946 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.766152 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.766220 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.767113 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:11.767114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:11.767115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:11.767121 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:11.769013 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.769386 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.769388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.769388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.769451 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.769669 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.769670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.769671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.769804 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:11.769887 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.770059 (+ 172us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.770137 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 22:42:11 dist-test-slave-g170 krb5kdc[25917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:11.797147 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.788648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47626 (local address 127.23.197.126:41343)
0805 22:42:11.788725 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.788728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.788795 (+ 67us) server_negotiation.cc:407] Connection header received
0805 22:42:11.788878 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.788879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.788897 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.788967 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:11.789755 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.790310 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.791165 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.791371 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.791475 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.794270 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:11.794272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:11.794273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:11.794281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:11.795892 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.796299 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.796311 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.796312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.796369 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:11.796573 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:11.796573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:11.796574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:11.796699 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:11.796829 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.796996 (+ 167us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.797062 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 22:42:11.799472 25980 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:11.801641 25980 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:11.809422 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.804937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58174 (local address 127.23.197.66:44307)
0805 22:42:11.805150 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.805155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.805165 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:11.805206 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.805208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.805225 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.805612 (+ 387us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:11.805701 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.806918 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.808066 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.808909 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.808993 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.809058 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.809242 (+ 184us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.809285 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":101,"threads_started":1}
I20260805 22:42:11.809432 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.804425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:44307 (local address 127.0.0.1:58174)
0805 22:42:11.804955 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.804965 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.805015 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.805413 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.805414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.805417 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:11.805496 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.805498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.806979 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.806980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.807976 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.807979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.809006 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.809007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.809109 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.809126 (+ 17us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.809227 (+ 101us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.809287 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":93,"threads_started":1}
I20260805 22:42:11.809439 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.804448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:36539 (local address 127.0.0.1:41200)
0805 22:42:11.805141 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.805160 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.805221 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.805456 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.805457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.805459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:11.805499 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.805501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.806937 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.806938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.808130 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.808131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.809114 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.809115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.809177 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.809203 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.809262 (+ 59us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.809338 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":5,"thread_start_us":109,"threads_started":1}
I20260805 22:42:11.809525 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.804633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57026 (local address 127.23.197.67:42071)
0805 22:42:11.804891 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.804897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.804911 (+ 14us) server_negotiation.cc:407] Connection header received
0805 22:42:11.804959 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.804961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.804982 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.805079 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:11.805457 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.806437 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.808244 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.808920 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.809015 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.809081 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.809350 (+ 269us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.809401 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":176,"threads_started":1}
I20260805 22:42:11.809698 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.804422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:42071 (local address 127.0.0.1:57026)
0805 22:42:11.804604 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.804669 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.804836 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.805180 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.805184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.805196 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:11.805355 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.805357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.806521 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.806522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.808128 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.808131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.809119 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.809120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.809187 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.809190 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.809252 (+ 62us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.809288 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":67,"threads_started":1}
I20260805 22:42:11.810376 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.804720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.23.197.65:36539)
0805 22:42:11.805145 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.805155 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.805209 (+ 54us) server_negotiation.cc:407] Connection header received
0805 22:42:11.805295 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.805297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.805312 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.805386 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:11.805577 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.806832 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.808224 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.808973 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.809032 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.809079 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.810226 (+ 1147us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.810277 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":153,"threads_started":1}
I20260805 22:42:11.810386 26584 tablet_service.cc:1467] Processing CreateTablet for tablet a54d46a721304dbf8be216f01adc05b8 (DEFAULT_TABLE table=test-table [id=739ce4fa7e3244a58f3d5d2a26ee1087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:11.810400 26386 tablet_service.cc:1467] Processing CreateTablet for tablet a54d46a721304dbf8be216f01adc05b8 (DEFAULT_TABLE table=test-table [id=739ce4fa7e3244a58f3d5d2a26ee1087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:11.810722 26584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54d46a721304dbf8be216f01adc05b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:11.810774 26386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54d46a721304dbf8be216f01adc05b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:11.811245 26172 tablet_service.cc:1467] Processing CreateTablet for tablet a54d46a721304dbf8be216f01adc05b8 (DEFAULT_TABLE table=test-table [id=739ce4fa7e3244a58f3d5d2a26ee1087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:11.811606 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54d46a721304dbf8be216f01adc05b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:11.813949 26713 tablet_bootstrap.cc:492] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: Bootstrap starting.
I20260805 22:42:11.814482 26714 tablet_bootstrap.cc:492] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: Bootstrap starting.
I20260805 22:42:11.814797 26713 tablet_bootstrap.cc:654] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:11.815231 26713 log.cc:824] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:11.815562 26714 tablet_bootstrap.cc:654] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:11.815991 26714 log.cc:824] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:11.816226 26713 tablet_bootstrap.cc:492] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: No bootstrap required, opened a new log
I20260805 22:42:11.816305 26713 ts_tablet_manager.cc:1397] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:11.816933 26714 tablet_bootstrap.cc:492] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: No bootstrap required, opened a new log
I20260805 22:42:11.817004 26714 ts_tablet_manager.cc:1397] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 22:42:11.818500 26713 raft_consensus.cc:348] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } }
I20260805 22:42:11.818681 26713 raft_consensus.cc:374] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:11.818722 26713 raft_consensus.cc:729] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc979ec85fc4bfe8eb205341672dea0, State: Initialized, Role: FOLLOWER
I20260805 22:42:11.818840 26713 consensus_queue.cc:260] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } }
I20260805 22:42:11.819135 26713 ts_tablet_manager.cc:1428] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:42:11.819197 26480 heartbeater.cc:503] Master 127.23.197.126:41343 was elected leader, sending a full tablet report...
I20260805 22:42:11.819271 26719 tablet_bootstrap.cc:492] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: Bootstrap starting.
I20260805 22:42:11.819265 26714 raft_consensus.cc:348] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } }
I20260805 22:42:11.819423 26714 raft_consensus.cc:374] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:11.819584 26714 raft_consensus.cc:729] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fef6214b78d4b64975bbae620d69f75, State: Initialized, Role: FOLLOWER
I20260805 22:42:11.819741 26714 consensus_queue.cc:260] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } }
I20260805 22:42:11.819983 26719 tablet_bootstrap.cc:654] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:11.820014 26714 ts_tablet_manager.cc:1428] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:42:11.820134 26677 heartbeater.cc:503] Master 127.23.197.126:41343 was elected leader, sending a full tablet report...
I20260805 22:42:11.820385 26719 log.cc:824] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:11.821211 26719 tablet_bootstrap.cc:492] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: No bootstrap required, opened a new log
I20260805 22:42:11.821310 26719 ts_tablet_manager.cc:1397] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260805 22:42:11.822783 26719 raft_consensus.cc:348] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } }
I20260805 22:42:11.822911 26719 raft_consensus.cc:374] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:11.822933 26719 raft_consensus.cc:729] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c347b293ab9f4e29807ccf325869ae96, State: Initialized, Role: FOLLOWER
I20260805 22:42:11.823055 26719 consensus_queue.cc:260] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } }
I20260805 22:42:11.823328 26719 ts_tablet_manager.cc:1428] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:11.823395 26282 heartbeater.cc:503] Master 127.23.197.126:41343 was elected leader, sending a full tablet report...
W20260805 22:42:11.843187 26482 tablet.cc:2369] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:42:11.897388 26285 tablet.cc:2369] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:42:11.947881 26722 raft_consensus.cc:482] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:11.948027 26722 raft_consensus.cc:504] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } }
I20260805 22:42:11.948431 26722 leader_election.cc:288] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fef6214b78d4b64975bbae620d69f75 (127.23.197.67:42071), c347b293ab9f4e29807ccf325869ae96 (127.23.197.65:36539)
I20260805 22:42:11.951515 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.948842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:59511 (local address 127.23.197.67:42071)
0805 22:42:11.948895 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.948898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.949013 (+ 115us) server_negotiation.cc:407] Connection header received
0805 22:42:11.949115 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.949116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.949124 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.949187 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:11.949356 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.949808 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.950785 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.951140 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.951225 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.951233 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.951397 (+ 164us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.951435 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260805 22:42:11.951515 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.948803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:33241 (local address 127.23.197.65:36539)
0805 22:42:11.948865 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.948867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.948879 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:42:11.948948 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.948949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.948960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.949038 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:11.949202 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.949686 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.950738 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.951116 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.951220 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.951231 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.951397 (+ 166us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.951440 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260805 22:42:11.951541 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.948697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:36539 (local address 127.23.197.66:33241)
0805 22:42:11.948759 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.948786 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.948882 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.949043 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.949044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.949046 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:11.949121 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.949123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.949785 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.949786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.950618 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.950620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.951227 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.951228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.951322 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.951324 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.951391 (+ 67us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.951438 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":20}
I20260805 22:42:11.951552 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.948708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:42071 (local address 127.23.197.66:59511)
0805 22:42:11.948924 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:11.948934 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:11.949024 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:11.949191 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:11.949192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:11.949194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:11.949280 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.949282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.949893 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.949894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.950680 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:11.950682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.951224 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.951225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:11.951318 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.951321 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:11.951390 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 22:42:11.951435 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":146,"threads_started":1}
I20260805 22:42:11.951903 26209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54d46a721304dbf8be216f01adc05b8" candidate_uuid: "9fc979ec85fc4bfe8eb205341672dea0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c347b293ab9f4e29807ccf325869ae96" is_pre_election: true
I20260805 22:42:11.951975 26613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54d46a721304dbf8be216f01adc05b8" candidate_uuid: "9fc979ec85fc4bfe8eb205341672dea0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fef6214b78d4b64975bbae620d69f75" is_pre_election: true
I20260805 22:42:11.952075 26209 raft_consensus.cc:2454] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fc979ec85fc4bfe8eb205341672dea0 in term 0.
I20260805 22:42:11.952162 26613 raft_consensus.cc:2454] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fc979ec85fc4bfe8eb205341672dea0 in term 0.
I20260805 22:42:11.952392 26320 leader_election.cc:302] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fc979ec85fc4bfe8eb205341672dea0, c347b293ab9f4e29807ccf325869ae96; no voters:
I20260805 22:42:11.952535 26722 raft_consensus.cc:2781] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:11.952592 26722 raft_consensus.cc:482] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:11.952616 26722 raft_consensus.cc:3037] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:11.953922 26722 raft_consensus.cc:504] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } }
I20260805 22:42:11.954142 26722 leader_election.cc:288] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [CANDIDATE]: Term 1 election: Requested vote from peers 7fef6214b78d4b64975bbae620d69f75 (127.23.197.67:42071), c347b293ab9f4e29807ccf325869ae96 (127.23.197.65:36539)
I20260805 22:42:11.954337 26613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54d46a721304dbf8be216f01adc05b8" candidate_uuid: "9fc979ec85fc4bfe8eb205341672dea0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fef6214b78d4b64975bbae620d69f75"
I20260805 22:42:11.954386 26209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54d46a721304dbf8be216f01adc05b8" candidate_uuid: "9fc979ec85fc4bfe8eb205341672dea0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c347b293ab9f4e29807ccf325869ae96"
I20260805 22:42:11.954448 26613 raft_consensus.cc:3037] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:11.954535 26209 raft_consensus.cc:3037] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:11.955276 26613 raft_consensus.cc:2454] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fc979ec85fc4bfe8eb205341672dea0 in term 1.
I20260805 22:42:11.955312 26209 raft_consensus.cc:2454] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fc979ec85fc4bfe8eb205341672dea0 in term 1.
I20260805 22:42:11.955503 26320 leader_election.cc:302] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fc979ec85fc4bfe8eb205341672dea0, c347b293ab9f4e29807ccf325869ae96; no voters:
I20260805 22:42:11.955598 26722 raft_consensus.cc:2781] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:11.955739 26722 raft_consensus.cc:686] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 LEADER]: Becoming Leader. State: Replica: 9fc979ec85fc4bfe8eb205341672dea0, State: Running, Role: LEADER
I20260805 22:42:11.955869 26722 consensus_queue.cc:237] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } }
I20260805 22:42:11.956820 25980 catalog_manager.cc:5495] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fc979ec85fc4bfe8eb205341672dea0 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "9fc979ec85fc4bfe8eb205341672dea0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:11.998426 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:11.995604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.23.197.66:44307)
0805 22:42:11.995677 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:11.995679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:11.995748 (+ 69us) server_negotiation.cc:407] Connection header received
0805 22:42:11.995786 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:11.995787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:11.995807 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:11.995865 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:11.996128 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.996648 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.997480 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:11.997684 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:11.997781 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:11.997971 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:11.998070 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:11.998134 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:11.998250 (+ 116us) server_negotiation.cc:299] Negotiation successful
0805 22:42:11.998271 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 22:42:12.000519 26209 raft_consensus.cc:1261] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Refusing update from remote peer 9fc979ec85fc4bfe8eb205341672dea0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:12.000555 26613 raft_consensus.cc:1261] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [term 1 FOLLOWER]: Refusing update from remote peer 9fc979ec85fc4bfe8eb205341672dea0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:12.000967 26731 consensus_queue.cc:1035] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:12.001099 26722 consensus_queue.cc:1035] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:12.003468 26736 mvcc.cc:205] Tried to move back new op lower bound from 7315332022271008768 to 7315332022091743232. Current Snapshot: MvccSnapshot[applied={T|T < 7315332022271008768}]
I20260805 22:42:12.007899 25980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:12.008083 25980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:12.008999 25980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c: Sending DeleteTablet for 3 replicas of tablet a54d46a721304dbf8be216f01adc05b8
I20260805 22:42:12.009506 26172 tablet_service.cc:1514] Processing DeleteTablet for tablet a54d46a721304dbf8be216f01adc05b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:12 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:41200
I20260805 22:42:12.009521 26386 tablet_service.cc:1514] Processing DeleteTablet for tablet a54d46a721304dbf8be216f01adc05b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:12 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:58174
I20260805 22:42:12.009529 26584 tablet_service.cc:1514] Processing DeleteTablet for tablet a54d46a721304dbf8be216f01adc05b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:12 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:57026
I20260805 22:42:12.009912 26751 tablet_replica.cc:330] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: stopping tablet replica
I20260805 22:42:12.009960 26754 tablet_replica.cc:330] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: stopping tablet replica
I20260805 22:42:12.010062 26754 raft_consensus.cc:2229] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:12.010071 25980 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260805 22:42:12.010087 26751 raft_consensus.cc:2229] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:12.010170 26754 raft_consensus.cc:2258] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260805 22:42:12.010242 25980 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:12.010344 26753 tablet_replica.cc:330] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: stopping tablet replica
I20260805 22:42:12.010437 26753 raft_consensus.cc:2229] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:12.010557 26753 raft_consensus.cc:2258] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:12.010581 26751 raft_consensus.cc:2258] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:12.011090 26754 ts_tablet_manager.cc:1905] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:12.011901 26386 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f (DEFAULT_TABLE table=test-table [id=a65e48b9e73b4b9cb9626e89fc8622d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:12.011907 26584 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f (DEFAULT_TABLE table=test-table [id=a65e48b9e73b4b9cb9626e89fc8622d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:12.012022 26386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:12.012045 26584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:12.013170 26753 ts_tablet_manager.cc:1905] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:12.013412 26713 tablet_bootstrap.cc:492] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: Bootstrap starting.
I20260805 22:42:12.013855 26713 tablet_bootstrap.cc:654] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:12.014214 26714 tablet_bootstrap.cc:492] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: Bootstrap starting.
I20260805 22:42:12.014631 26714 tablet_bootstrap.cc:654] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:12.014956 26753 ts_tablet_manager.cc:1918] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:12.015034 26753 log.cc:1196] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/a54d46a721304dbf8be216f01adc05b8
I20260805 22:42:12.015309 26753 ts_tablet_manager.cc:1939] T a54d46a721304dbf8be216f01adc05b8 P 7fef6214b78d4b64975bbae620d69f75: Deleting consensus metadata
I20260805 22:42:12.015427 26172 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f (DEFAULT_TABLE table=test-table [id=a65e48b9e73b4b9cb9626e89fc8622d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:12.015476 26714 tablet_bootstrap.cc:492] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: No bootstrap required, opened a new log
I20260805 22:42:12.015525 26714 ts_tablet_manager.cc:1397] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:12.015542 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:12.015722 26714 raft_consensus.cc:348] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.015808 26714 raft_consensus.cc:374] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:12.015833 26714 raft_consensus.cc:729] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fef6214b78d4b64975bbae620d69f75, State: Initialized, Role: FOLLOWER
I20260805 22:42:12.015853 26754 ts_tablet_manager.cc:1918] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:12.015885 26714 consensus_queue.cc:260] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.015918 26754 log.cc:1196] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/a54d46a721304dbf8be216f01adc05b8
I20260805 22:42:12.015981 26714 ts_tablet_manager.cc:1428] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:12.016182 26754 ts_tablet_manager.cc:1939] T a54d46a721304dbf8be216f01adc05b8 P 9fc979ec85fc4bfe8eb205341672dea0: Deleting consensus metadata
I20260805 22:42:12.016287 25961 catalog_manager.cc:4841] TS 7fef6214b78d4b64975bbae620d69f75 (127.23.197.67:42071): tablet a54d46a721304dbf8be216f01adc05b8 (table test-table [id=739ce4fa7e3244a58f3d5d2a26ee1087]) successfully deleted
I20260805 22:42:12.017124 26719 tablet_bootstrap.cc:492] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: Bootstrap starting.
I20260805 22:42:12.017313 25960 catalog_manager.cc:4841] TS 9fc979ec85fc4bfe8eb205341672dea0 (127.23.197.66:44307): tablet a54d46a721304dbf8be216f01adc05b8 (table test-table [id=739ce4fa7e3244a58f3d5d2a26ee1087]) successfully deleted
I20260805 22:42:12.017534 26719 tablet_bootstrap.cc:654] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:12.018141 26751 ts_tablet_manager.cc:1905] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:12.018288 26719 tablet_bootstrap.cc:492] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: No bootstrap required, opened a new log
I20260805 22:42:12.018354 26719 ts_tablet_manager.cc:1397] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:12.018525 26719 raft_consensus.cc:348] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.018613 26719 raft_consensus.cc:374] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:12.018640 26719 raft_consensus.cc:729] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c347b293ab9f4e29807ccf325869ae96, State: Initialized, Role: FOLLOWER
I20260805 22:42:12.018687 26719 consensus_queue.cc:260] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.018798 26719 ts_tablet_manager.cc:1428] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:12.019979 26751 ts_tablet_manager.cc:1918] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:12.020061 26751 log.cc:1196] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/a54d46a721304dbf8be216f01adc05b8
I20260805 22:42:12.020341 26713 tablet_bootstrap.cc:492] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: No bootstrap required, opened a new log
I20260805 22:42:12.020402 26713 ts_tablet_manager.cc:1397] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260805 22:42:12.020589 26713 raft_consensus.cc:348] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.020686 26713 raft_consensus.cc:374] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:12.020713 26713 raft_consensus.cc:729] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc979ec85fc4bfe8eb205341672dea0, State: Initialized, Role: FOLLOWER
I20260805 22:42:12.020714 26751 ts_tablet_manager.cc:1939] T a54d46a721304dbf8be216f01adc05b8 P c347b293ab9f4e29807ccf325869ae96: Deleting consensus metadata
I20260805 22:42:12.020790 26713 consensus_queue.cc:260] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.020896 26713 ts_tablet_manager.cc:1428] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:12.021198 25962 catalog_manager.cc:4841] TS c347b293ab9f4e29807ccf325869ae96 (127.23.197.65:36539): tablet a54d46a721304dbf8be216f01adc05b8 (table test-table [id=739ce4fa7e3244a58f3d5d2a26ee1087]) successfully deleted
I20260805 22:42:12.135558 26725 raft_consensus.cc:482] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:12.135667 26725 raft_consensus.cc:504] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.136201 26725 leader_election.cc:288] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c347b293ab9f4e29807ccf325869ae96 (127.23.197.65:36539), 9fc979ec85fc4bfe8eb205341672dea0 (127.23.197.66:44307)
I20260805 22:42:12.139374 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.136478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:52225 (local address 127.23.197.65:36539)
0805 22:42:12.136546 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:12.136548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:12.136559 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:12.136593 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:12.136594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:12.136611 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:12.136684 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:12.136909 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.137460 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.138528 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.139002 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.139087 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.139097 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:12.139267 (+ 170us) server_negotiation.cc:299] Negotiation successful
0805 22:42:12.139306 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 22:42:12.139374 26523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.136394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:36539 (local address 127.23.197.67:52225)
0805 22:42:12.136458 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:12.136472 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:12.136531 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:12.136718 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:12.136719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:12.136721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:12.136821 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.136824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.137569 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.137570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.138428 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.138430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.139085 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.139086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.139176 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.139180 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:12.139246 (+ 66us) client_negotiation.cc:240] Negotiation successful
0805 22:42:12.139290 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":30}
I20260805 22:42:12.139484 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.136508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:52357 (local address 127.23.197.66:44307)
0805 22:42:12.136559 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:12.136562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:12.136663 (+ 101us) server_negotiation.cc:407] Connection header received
0805 22:42:12.136736 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:12.136737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:12.136748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:12.136843 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:12.137010 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.137483 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.138603 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.139025 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.139114 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.139125 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:12.139348 (+ 223us) server_negotiation.cc:299] Negotiation successful
0805 22:42:12.139395 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260805 22:42:12.139712 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.136420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:44307 (local address 127.23.197.67:52357)
0805 22:42:12.136582 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:12.136591 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:12.136679 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:12.136847 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:12.136848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:12.136849 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:12.136938 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.136940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.137592 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.137593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.138498 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.138501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.139136 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.139137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.139215 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.139218 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:12.139323 (+ 105us) client_negotiation.cc:240] Negotiation successful
0805 22:42:12.139371 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":27,"thread_start_us":100,"threads_started":1}
I20260805 22:42:12.139663 26209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7adc5d952a4bdfb1c8f6a873ddad3f" candidate_uuid: "7fef6214b78d4b64975bbae620d69f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c347b293ab9f4e29807ccf325869ae96" is_pre_election: true
I20260805 22:42:12.139770 26209 raft_consensus.cc:2454] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fef6214b78d4b64975bbae620d69f75 in term 0.
I20260805 22:42:12.139976 26511 leader_election.cc:302] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fef6214b78d4b64975bbae620d69f75, c347b293ab9f4e29807ccf325869ae96; no voters:
I20260805 22:42:12.139993 26415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7adc5d952a4bdfb1c8f6a873ddad3f" candidate_uuid: "7fef6214b78d4b64975bbae620d69f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc979ec85fc4bfe8eb205341672dea0" is_pre_election: true
I20260805 22:42:12.140128 26725 raft_consensus.cc:2781] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:12.140136 26415 raft_consensus.cc:2454] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fef6214b78d4b64975bbae620d69f75 in term 0.
I20260805 22:42:12.140167 26725 raft_consensus.cc:482] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:12.140197 26725 raft_consensus.cc:3037] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:12.141003 26725 raft_consensus.cc:504] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.141179 26725 leader_election.cc:288] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [CANDIDATE]: Term 1 election: Requested vote from peers c347b293ab9f4e29807ccf325869ae96 (127.23.197.65:36539), 9fc979ec85fc4bfe8eb205341672dea0 (127.23.197.66:44307)
I20260805 22:42:12.141337 26415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7adc5d952a4bdfb1c8f6a873ddad3f" candidate_uuid: "7fef6214b78d4b64975bbae620d69f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc979ec85fc4bfe8eb205341672dea0"
I20260805 22:42:12.141381 26209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7adc5d952a4bdfb1c8f6a873ddad3f" candidate_uuid: "7fef6214b78d4b64975bbae620d69f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c347b293ab9f4e29807ccf325869ae96"
I20260805 22:42:12.141443 26415 raft_consensus.cc:3037] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:12.141454 26209 raft_consensus.cc:3037] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:12.142100 26209 raft_consensus.cc:2454] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fef6214b78d4b64975bbae620d69f75 in term 1.
I20260805 22:42:12.142199 26415 raft_consensus.cc:2454] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fef6214b78d4b64975bbae620d69f75 in term 1.
I20260805 22:42:12.142264 26511 leader_election.cc:302] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fef6214b78d4b64975bbae620d69f75, c347b293ab9f4e29807ccf325869ae96; no voters:
I20260805 22:42:12.142379 26725 raft_consensus.cc:2781] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:12.142577 26725 raft_consensus.cc:686] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 1 LEADER]: Becoming Leader. State: Replica: 7fef6214b78d4b64975bbae620d69f75, State: Running, Role: LEADER
I20260805 22:42:12.142648 26725 consensus_queue.cc:237] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.143455 25979 catalog_manager.cc:5495] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fef6214b78d4b64975bbae620d69f75 (127.23.197.67). New cstate: current_term: 1 leader_uuid: "7fef6214b78d4b64975bbae620d69f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } health_report { overall_health: HEALTHY } } }
Aug 05 22:42:12 dist-test-slave-g170 krb5kdc[25917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:12.201793 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.193762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.23.197.126:41343)
0805 22:42:12.193842 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:12.193845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:12.193980 (+ 135us) server_negotiation.cc:407] Connection header received
0805 22:42:12.194024 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:12.194025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:12.194035 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:12.194090 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:12.194803 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.195446 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.196337 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.196520 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.196595 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.198873 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:12.198875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:12.198875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:12.198882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:12.200359 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:12.200929 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:12.200930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:12.200930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:12.200993 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:12.201213 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:12.201213 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:12.201214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:12.201330 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:12.201408 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:12.201569 (+ 161us) server_negotiation.cc:299] Negotiation successful
0805 22:42:12.201680 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 22:42:12.201862 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.193651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41343 (local address 127.0.0.1:47634)
0805 22:42:12.193890 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:12.193902 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:12.193978 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:12.194111 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:12.194112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:12.194607 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:12.194669 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.194671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.195631 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.195633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.196229 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.196231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.196592 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.196593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.196644 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.197355 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:12.197360 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:12.198775 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:12.200554 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:12.200562 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:12.200565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:12.200854 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:12.201084 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:12.201084 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:12.201085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:12.201149 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:12.201397 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:12.201405 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:12.201456 (+ 51us) client_negotiation.cc:769] Sending connection context
0805 22:42:12.201544 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 22:42:12.201741 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":93,"thread_start_us":132,"threads_started":1}
I20260805 22:42:12.204425 25979 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:47634:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260805 22:42:12.204797 25979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:12.206497 26172 tablet_service.cc:1467] Processing CreateTablet for tablet 002b342edc2744169f61a52a33fa3bce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=31b1499cb207414389cccb9b3acd0616]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 22:42:12.206525 26386 tablet_service.cc:1467] Processing CreateTablet for tablet 002b342edc2744169f61a52a33fa3bce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=31b1499cb207414389cccb9b3acd0616]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 22:42:12.206737 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002b342edc2744169f61a52a33fa3bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:12.206755 26584 tablet_service.cc:1467] Processing CreateTablet for tablet 002b342edc2744169f61a52a33fa3bce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=31b1499cb207414389cccb9b3acd0616]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 22:42:12.206811 26386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002b342edc2744169f61a52a33fa3bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:12.206897 26584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002b342edc2744169f61a52a33fa3bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:12.208185 26719 tablet_bootstrap.cc:492] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: Bootstrap starting.
I20260805 22:42:12.208416 26714 tablet_bootstrap.cc:492] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: Bootstrap starting.
I20260805 22:42:12.208716 26719 tablet_bootstrap.cc:654] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:12.208877 26713 tablet_bootstrap.cc:492] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: Bootstrap starting.
I20260805 22:42:12.208915 26714 tablet_bootstrap.cc:654] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:12.209225 26713 tablet_bootstrap.cc:654] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:12.209512 26719 tablet_bootstrap.cc:492] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: No bootstrap required, opened a new log
I20260805 22:42:12.209565 26719 ts_tablet_manager.cc:1397] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:12.209725 26719 raft_consensus.cc:348] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.209789 26719 raft_consensus.cc:374] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:12.209805 26719 raft_consensus.cc:729] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c347b293ab9f4e29807ccf325869ae96, State: Initialized, Role: FOLLOWER
I20260805 22:42:12.209851 26719 consensus_queue.cc:260] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.209951 26719 ts_tablet_manager.cc:1428] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:12.210069 26728 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.210214 26728 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:12.210364 26714 tablet_bootstrap.cc:492] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: No bootstrap required, opened a new log
I20260805 22:42:12.210422 26714 ts_tablet_manager.cc:1397] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260805 22:42:12.210430 26713 tablet_bootstrap.cc:492] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: No bootstrap required, opened a new log
I20260805 22:42:12.210479 26713 ts_tablet_manager.cc:1397] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260805 22:42:12.210587 26713 raft_consensus.cc:348] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.210615 26714 raft_consensus.cc:348] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.210668 26714 raft_consensus.cc:374] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:12.210667 26713 raft_consensus.cc:374] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:12.210692 26713 raft_consensus.cc:729] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc979ec85fc4bfe8eb205341672dea0, State: Initialized, Role: FOLLOWER
I20260805 22:42:12.210683 26714 raft_consensus.cc:729] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fef6214b78d4b64975bbae620d69f75, State: Initialized, Role: FOLLOWER
I20260805 22:42:12.210753 26713 consensus_queue.cc:260] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.210785 26714 consensus_queue.cc:260] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.210860 26713 ts_tablet_manager.cc:1428] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:12.210901 26714 ts_tablet_manager.cc:1428] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:12.210953 26722 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.210958 26725 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.211019 26722 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:12.211041 26725 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: This TxnStatusTablet replica's current role is: FOLLOWER
W20260805 22:42:12.259943 26678 tablet.cc:2369] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:42:12.442742 26725 raft_consensus.cc:482] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:12.442862 26725 raft_consensus.cc:504] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.443073 26725 leader_election.cc:288] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c347b293ab9f4e29807ccf325869ae96 (127.23.197.65:36539), 9fc979ec85fc4bfe8eb205341672dea0 (127.23.197.66:44307)
I20260805 22:42:12.443420 26415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002b342edc2744169f61a52a33fa3bce" candidate_uuid: "7fef6214b78d4b64975bbae620d69f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc979ec85fc4bfe8eb205341672dea0" is_pre_election: true
I20260805 22:42:12.443420 26209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002b342edc2744169f61a52a33fa3bce" candidate_uuid: "7fef6214b78d4b64975bbae620d69f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c347b293ab9f4e29807ccf325869ae96" is_pre_election: true
I20260805 22:42:12.443565 26415 raft_consensus.cc:2454] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fef6214b78d4b64975bbae620d69f75 in term 0.
I20260805 22:42:12.443565 26209 raft_consensus.cc:2454] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fef6214b78d4b64975bbae620d69f75 in term 0.
I20260805 22:42:12.443843 26508 leader_election.cc:302] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fef6214b78d4b64975bbae620d69f75, 9fc979ec85fc4bfe8eb205341672dea0; no voters:
I20260805 22:42:12.443992 26725 raft_consensus.cc:2781] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:12.444099 26725 raft_consensus.cc:482] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:12.444139 26725 raft_consensus.cc:3037] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:12.445000 26725 raft_consensus.cc:504] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.445187 26725 leader_election.cc:288] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [CANDIDATE]: Term 1 election: Requested vote from peers c347b293ab9f4e29807ccf325869ae96 (127.23.197.65:36539), 9fc979ec85fc4bfe8eb205341672dea0 (127.23.197.66:44307)
I20260805 22:42:12.445376 26209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002b342edc2744169f61a52a33fa3bce" candidate_uuid: "7fef6214b78d4b64975bbae620d69f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c347b293ab9f4e29807ccf325869ae96"
I20260805 22:42:12.445390 26415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002b342edc2744169f61a52a33fa3bce" candidate_uuid: "7fef6214b78d4b64975bbae620d69f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc979ec85fc4bfe8eb205341672dea0"
I20260805 22:42:12.445472 26209 raft_consensus.cc:3037] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:12.445477 26415 raft_consensus.cc:3037] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:12.446512 26415 raft_consensus.cc:2454] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fef6214b78d4b64975bbae620d69f75 in term 1.
I20260805 22:42:12.446683 26209 raft_consensus.cc:2454] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fef6214b78d4b64975bbae620d69f75 in term 1.
I20260805 22:42:12.446790 26508 leader_election.cc:302] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fef6214b78d4b64975bbae620d69f75, 9fc979ec85fc4bfe8eb205341672dea0; no voters:
I20260805 22:42:12.446897 26725 raft_consensus.cc:2781] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:12.447068 26725 raft_consensus.cc:686] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [term 1 LEADER]: Becoming Leader. State: Replica: 7fef6214b78d4b64975bbae620d69f75, State: Running, Role: LEADER
I20260805 22:42:12.447160 26725 consensus_queue.cc:237] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } }
I20260805 22:42:12.447496 26768 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: TxnStatusTablet state changed. Reason: New leader 7fef6214b78d4b64975bbae620d69f75. Latest consensus state: current_term: 1 leader_uuid: "7fef6214b78d4b64975bbae620d69f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.447613 26768 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: This TxnStatusTablet replica's current role is: LEADER
I20260805 22:42:12.447837 26785 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260805 22:42:12.447988 26785 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260805 22:42:12.448134 25978 catalog_manager.cc:5495] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fef6214b78d4b64975bbae620d69f75 (127.23.197.67). New cstate: current_term: 1 leader_uuid: "7fef6214b78d4b64975bbae620d69f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } health_report { overall_health: HEALTHY } } }
I20260805 22:42:12.497802 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.494173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:42071 (local address 127.0.0.1:57040)
0805 22:42:12.494293 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:12.494317 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:12.494373 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:12.494761 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:12.494763 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:12.494765 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:12.494839 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.494842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.495832 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.495834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.496580 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.496584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.497127 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.497129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.497203 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.497222 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:12.497576 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:12.497590 (+ 14us) client_negotiation.cc:769] Sending connection context
0805 22:42:12.497658 (+ 68us) client_negotiation.cc:240] Negotiation successful
0805 22:42:12.497718 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260805 22:42:12.497815 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.494325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.23.197.67:42071)
0805 22:42:12.494477 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:12.494479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:12.494489 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:12.494537 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:12.494537 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:12.494641 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:12.494753 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:12.494935 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.495720 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.496696 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.496998 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.497114 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.497353 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:12.497489 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:12.497603 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:12.497666 (+ 63us) server_negotiation.cc:299] Negotiation successful
0805 22:42:12.497694 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260805 22:42:12.499363 26209 raft_consensus.cc:1261] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Refusing update from remote peer 7fef6214b78d4b64975bbae620d69f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:12.499480 26415 raft_consensus.cc:1261] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 FOLLOWER]: Refusing update from remote peer 7fef6214b78d4b64975bbae620d69f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:12.499699 26768 consensus_queue.cc:1035] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:12.499835 26725 consensus_queue.cc:1035] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:12.500694 26722 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: TxnStatusTablet state changed. Reason: New leader 7fef6214b78d4b64975bbae620d69f75. Latest consensus state: current_term: 1 leader_uuid: "7fef6214b78d4b64975bbae620d69f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.500825 26722 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:12.500723 26728 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: TxnStatusTablet state changed. Reason: New leader 7fef6214b78d4b64975bbae620d69f75. Latest consensus state: current_term: 1 leader_uuid: "7fef6214b78d4b64975bbae620d69f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.500867 26728 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:12.501605 26738 mvcc.cc:205] Tried to move back new op lower bound from 7315332024314974208 to 7315332024104058880. Current Snapshot: MvccSnapshot[applied={T|T < 7315332024314974208}]
I20260805 22:42:12.501700 26725 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7fef6214b78d4b64975bbae620d69f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.501785 26725 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: This TxnStatusTablet replica's current role is: LEADER
I20260805 22:42:12.501999 26737 mvcc.cc:205] Tried to move back new op lower bound from 7315332024314974208 to 7315332024104058880. Current Snapshot: MvccSnapshot[applied={T|T < 7315332024314974208}]
I20260805 22:42:12.502063 26768 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7fef6214b78d4b64975bbae620d69f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.502125 26768 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P 7fef6214b78d4b64975bbae620d69f75: This TxnStatusTablet replica's current role is: LEADER
I20260805 22:42:12.502519 26722 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7fef6214b78d4b64975bbae620d69f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.502561 26728 tablet_replica.cc:439] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7fef6214b78d4b64975bbae620d69f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 } } peers { permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 } } peers { permanent_uuid: "7fef6214b78d4b64975bbae620d69f75" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 42071 } } }
I20260805 22:42:12.502626 26722 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P 9fc979ec85fc4bfe8eb205341672dea0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:12.502638 26728 tablet_replica.cc:442] T 002b342edc2744169f61a52a33fa3bce P c347b293ab9f4e29807ccf325869ae96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:12.508463 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.505430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57048 (local address 127.23.197.67:42071)
0805 22:42:12.505469 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:12.505470 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:12.505564 (+ 94us) server_negotiation.cc:407] Connection header received
0805 22:42:12.505643 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:12.505644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:12.505651 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:12.505698 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:12.505854 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.506401 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.507759 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.507981 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.508047 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.508230 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:12.508258 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:12.508302 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:12.508382 (+ 80us) server_negotiation.cc:299] Negotiation successful
0805 22:42:12.508400 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260805 22:42:12.514358 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.511563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:57863 (local address 127.23.197.67:42071)
0805 22:42:12.511603 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:12.511605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:12.511875 (+ 270us) server_negotiation.cc:407] Connection header received
0805 22:42:12.511902 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:12.511903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:12.511912 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:12.511974 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:12.512111 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.512576 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.513671 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.513863 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.513923 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.514106 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:12.514131 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:12.514172 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:12.514245 (+ 73us) server_negotiation.cc:299] Negotiation successful
0805 22:42:12.514274 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:42:12.514591 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.511527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:42071 (local address 127.23.197.67:57863)
0805 22:42:12.511716 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:12.511729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:12.511797 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:12.511993 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:12.511994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:12.511995 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:12.512040 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.512042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.512662 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.512662 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.513585 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:12.513587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.513944 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.513945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:12.514003 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.514016 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:12.514189 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:12.514190 (+ 1us) client_negotiation.cc:769] Sending connection context
0805 22:42:12.514243 (+ 53us) client_negotiation.cc:240] Negotiation successful
0805 22:42:12.514293 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260805 22:42:12.517040 26209 raft_consensus.cc:1261] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Refusing update from remote peer 7fef6214b78d4b64975bbae620d69f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:12.517167 26415 raft_consensus.cc:1261] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 FOLLOWER]: Refusing update from remote peer 7fef6214b78d4b64975bbae620d69f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:12.517256 26768 consensus_queue.cc:1035] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c347b293ab9f4e29807ccf325869ae96" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:12.517431 26768 consensus_queue.cc:1035] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fc979ec85fc4bfe8eb205341672dea0" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:12.530823 25980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:12.530943 25980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:12.531599 25980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b36b33e447343a98db2b15903f38c9c: Sending DeleteTablet for 3 replicas of tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f
I20260805 22:42:12.531852 26386 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:12 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:58174
I20260805 22:42:12.531889 26172 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:12 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:41200
I20260805 22:42:12.532076 26803 tablet_replica.cc:330] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: stopping tablet replica
I20260805 22:42:12.532130 26803 raft_consensus.cc:2229] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:12.532164 26803 raft_consensus.cc:2258] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:12.532133 26584 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:12 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:57026
I20260805 22:42:12.532346 26804 tablet_replica.cc:330] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: stopping tablet replica
I20260805 22:42:12.532409 26804 raft_consensus.cc:2229] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:12.532444 26804 raft_consensus.cc:2258] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:12.532433 26803 ts_tablet_manager.cc:1905] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:12.533128 26804 ts_tablet_manager.cc:1905] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:12.534391 26803 ts_tablet_manager.cc:1918] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:12.534461 26803 log.cc:1196] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/8e7adc5d952a4bdfb1c8f6a873ddad3f
I20260805 22:42:12.534603 26804 ts_tablet_manager.cc:1918] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:12.534690 26803 ts_tablet_manager.cc:1939] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 9fc979ec85fc4bfe8eb205341672dea0: Deleting consensus metadata
I20260805 22:42:12.534698 26804 log.cc:1196] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/8e7adc5d952a4bdfb1c8f6a873ddad3f
I20260805 22:42:12.535018 26804 ts_tablet_manager.cc:1939] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P c347b293ab9f4e29807ccf325869ae96: Deleting consensus metadata
I20260805 22:42:12.535073 25960 catalog_manager.cc:4841] TS 9fc979ec85fc4bfe8eb205341672dea0 (127.23.197.66:44307): tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f (table test-table [id=a65e48b9e73b4b9cb9626e89fc8622d1]) successfully deleted
I20260805 22:42:12.535372 26805 tablet_replica.cc:330] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: stopping tablet replica
I20260805 22:42:12.535391 25962 catalog_manager.cc:4841] TS c347b293ab9f4e29807ccf325869ae96 (127.23.197.65:36539): tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f (table test-table [id=a65e48b9e73b4b9cb9626e89fc8622d1]) successfully deleted
I20260805 22:42:12.535446 26805 raft_consensus.cc:2229] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:12.535554 26805 raft_consensus.cc:2258] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:12.535823 26805 ts_tablet_manager.cc:1905] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:12.537618 26805 ts_tablet_manager.cc:1918] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:12.537695 26805 log.cc:1196] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/8e7adc5d952a4bdfb1c8f6a873ddad3f
I20260805 22:42:12.538031 26805 ts_tablet_manager.cc:1939] T 8e7adc5d952a4bdfb1c8f6a873ddad3f P 7fef6214b78d4b64975bbae620d69f75: Deleting consensus metadata
I20260805 22:42:12.538390 25961 catalog_manager.cc:4841] TS 7fef6214b78d4b64975bbae620d69f75 (127.23.197.67:42071): tablet 8e7adc5d952a4bdfb1c8f6a873ddad3f (table test-table [id=a65e48b9e73b4b9cb9626e89fc8622d1]) successfully deleted
Aug 05 22:42:12 dist-test-slave-g170 krb5kdc[25917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969731, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:12.546893 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.533238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41202 (local address 127.23.197.65:36539)
0805 22:42:12.533294 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:12.533296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:12.536983 (+ 3687us) server_negotiation.cc:407] Connection header received
0805 22:42:12.537042 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:12.537043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:12.537058 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:12.537151 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:12.538057 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.538844 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.539644 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.539926 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.540008 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.543422 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:12.543433 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:12.543434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:12.543442 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:12.545589 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:12.546006 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:12.546011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:12.546011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:12.546123 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:12.546329 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:12.546330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:12.546331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:12.546516 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:12.546674 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:12.546717 (+ 43us) server_negotiation.cc:299] Negotiation successful
0805 22:42:12.546801 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260805 22:42:12.547168 26274 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41202
I20260805 22:42:12.554876 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:12.548236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47642 (local address 127.23.197.126:41343)
0805 22:42:12.548298 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:12.548300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:12.548346 (+ 46us) server_negotiation.cc:407] Connection header received
0805 22:42:12.548400 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:12.548401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:12.548413 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:12.548480 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:12.549036 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.549794 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.550490 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:12.550794 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:12.550874 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:12.552013 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:12.552014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:12.552015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:12.552022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:12.553629 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:12.553958 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:12.553959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:12.553959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:12.554023 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:12.554299 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:12.554300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:12.554301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:12.554430 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:12.554540 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:12.554689 (+ 149us) server_negotiation.cc:299] Negotiation successful
0805 22:42:12.554772 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260805 22:42:12.555131 25980 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47642
I20260805 22:42:12.555830 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 26095
I20260805 22:42:12.571457 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 26295
I20260805 22:42:12.586675 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 26491
I20260805 22:42:12.602104 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 25938
2026-08-05T22:42:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2908 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:12 dist-test-slave-g170 krb5kdc[26813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:12 dist-test-slave-g170 krb5kdc[26813](info): set up 2 sockets
Aug 05 22:42:12 dist-test-slave-g170 krb5kdc[26813](info): commencing operation
krb5kdc: starting...
W20260805 22:42:13.864075 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:13 dist-test-slave-g170 krb5kdc[26813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:14.007567 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33671
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:33671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:14.094234 26831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:14.094451 26831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:14.094475 26831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:14.095566 26831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:14.095603 26831 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260805 22:42:14.095616 26831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:14.095636 26831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:14.095649 26831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:14.095661 26831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:14.097115 26831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:33671
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:14.097318 26831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:14.097502 26831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:14.103097 26839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:14.103158 26840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:14.103133 26842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:14.103435 26831 server_base.cc:1034] running on GCE node
I20260805 22:42:14.103703 26831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:14.104035 26831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:14.105213 26831 hybrid_clock.cc:648] HybridClock initialized: now 1785969734105190 us; error 42 us; skew 500 ppm
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:14.107852 26831 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:14.108330 26831 webserver.cc:460] Webserver started at http://127.23.197.126:43037/ using document root <none> and password file <none>
I20260805 22:42:14.108596 26831 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:14.108669 26831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:14.108872 26831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:14.110080 26831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "343a319865b249b2a2a9e7cd009f2ea4"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "5e5298d4ad4946d18d054e374bb4264c"
server_key_iv: "10cc12942accfd2cfbc8ca7156f81067"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.110474 26831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "343a319865b249b2a2a9e7cd009f2ea4"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "5e5298d4ad4946d18d054e374bb4264c"
server_key_iv: "10cc12942accfd2cfbc8ca7156f81067"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.111889 26831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:14.112823 26855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:14.113019 26831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:14.113168 26831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "343a319865b249b2a2a9e7cd009f2ea4"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "5e5298d4ad4946d18d054e374bb4264c"
server_key_iv: "10cc12942accfd2cfbc8ca7156f81067"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.113258 26831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:14.128625 26831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:14.130810 26831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:14.130990 26831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:14.136264 26831 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:33671
I20260805 22:42:14.136358 26945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:33671 every 8 connection(s)
I20260805 22:42:14.136735 26831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:14.137655 26946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:14.140252 26946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: Bootstrap starting.
I20260805 22:42:14.140959 26946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:14.141309 26946 log.cc:824] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:14.142158 26946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: No bootstrap required, opened a new log
I20260805 22:42:14.143723 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 26831
I20260805 22:42:14.143836 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:14.144026 24341 external_mini_cluster.cc:1428] Setting key 7478b2fe87636cfba72f641d619e0c66
I20260805 22:42:14.145048 26946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343a319865b249b2a2a9e7cd009f2ea4" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33671 } }
I20260805 22:42:14.145212 26946 raft_consensus.cc:374] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:14.145279 26946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343a319865b249b2a2a9e7cd009f2ea4, State: Initialized, Role: FOLLOWER
I20260805 22:42:14.145447 26946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343a319865b249b2a2a9e7cd009f2ea4" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33671 } }
I20260805 22:42:14.145551 26946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:14.145592 26946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:14.145654 26946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:14.146461 26946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343a319865b249b2a2a9e7cd009f2ea4" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33671 } }
I20260805 22:42:14.146566 26946 leader_election.cc:302] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 343a319865b249b2a2a9e7cd009f2ea4; no voters:
I20260805 22:42:14.146800 26946 leader_election.cc:288] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:14.146848 26952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:14.146991 26952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [term 1 LEADER]: Becoming Leader. State: Replica: 343a319865b249b2a2a9e7cd009f2ea4, State: Running, Role: LEADER
I20260805 22:42:14.147119 26946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:14.147104 26952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343a319865b249b2a2a9e7cd009f2ea4" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33671 } }
I20260805 22:42:14.147424 26953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "343a319865b249b2a2a9e7cd009f2ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343a319865b249b2a2a9e7cd009f2ea4" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33671 } } }
I20260805 22:42:14.147516 26953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:14.147414 26952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 343a319865b249b2a2a9e7cd009f2ea4. Latest consensus state: current_term: 1 leader_uuid: "343a319865b249b2a2a9e7cd009f2ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343a319865b249b2a2a9e7cd009f2ea4" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33671 } } }
I20260805 22:42:14.147727 26952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4 [sys.catalog]: This master's current role is: LEADER
W20260805 22:42:14.149266 26975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:42:14.149327 26975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:42:14.149410 26959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:14.149582 26959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:14.151409 26959 catalog_manager.cc:1347] Generated new cluster ID: 07ac6392cf1d45019dc75f41130752aa
I20260805 22:42:14.151470 26959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:14.156526 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.144693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50366 (local address 127.23.197.126:33671)
0805 22:42:14.144930 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.144940 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.144967 (+ 27us) server_negotiation.cc:407] Connection header received
0805 22:42:14.145077 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.145083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.145134 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.145292 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:14.146186 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.147503 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.148512 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.149158 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.149385 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.152434 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:14.152440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:14.152440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:14.152453 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:14.154721 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.155190 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.155193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.155193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.155341 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.155561 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.155562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.155563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.155906 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:14.156087 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.156157 (+ 70us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.156315 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":138,"threads_started":1}
I20260805 22:42:14.170928 26959 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:14.171481 26959 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:14.178973 26959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: Generated new TSK 0
I20260805 22:42:14.179214 26959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:14.215790 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33671
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:14.304963 26984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:14.305176 26984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:14.305202 26984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:14.306309 26984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:14.306344 26984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:14.306365 26984 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260805 22:42:14.306376 26984 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 22:42:14.306393 26984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:14.308645 26984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.197.126:33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:14.308926 26984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:14.309207 26984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:14.315076 26990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:14.315084 26989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:14.315297 26984 server_base.cc:1034] running on GCE node
W20260805 22:42:14.315421 26992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:14.315673 26984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:14.316007 26984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:14.317204 26984 hybrid_clock.cc:648] HybridClock initialized: now 1785969734317169 us; error 51 us; skew 500 ppm
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:14.320214 26984 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:14.320710 26984 webserver.cc:460] Webserver started at http://127.23.197.65:39475/ using document root <none> and password file <none>
I20260805 22:42:14.320971 26984 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:14.321020 26984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:14.321172 26984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:14.322290 26984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "736645fea67f4798ae7f44251332f7d6"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "7e6ce8945637af25b9099f661466a227"
server_key_iv: "b3e0f74e6ff23c066b458a02c5dee3a6"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.322737 26984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "736645fea67f4798ae7f44251332f7d6"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "7e6ce8945637af25b9099f661466a227"
server_key_iv: "b3e0f74e6ff23c066b458a02c5dee3a6"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.324359 26984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:14.325167 27007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:14.325342 26984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:14.325446 26984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "736645fea67f4798ae7f44251332f7d6"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "7e6ce8945637af25b9099f661466a227"
server_key_iv: "b3e0f74e6ff23c066b458a02c5dee3a6"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.325529 26984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:14.336233 26984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:14.338814 26984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:14.338987 26984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:14.339541 26984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:14.339605 26984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:14.339658 26984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:14.339682 26984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:14.346997 26984 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:41203
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:14.347128 27175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:41203 every 8 connection(s)
I20260805 22:42:14.347442 26984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:14.350544 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.340117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:43945 (local address 127.23.197.126:33671)
0805 22:42:14.340162 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.340165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.340387 (+ 222us) server_negotiation.cc:407] Connection header received
0805 22:42:14.340966 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.340968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.340987 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.341064 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:14.341830 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.342681 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.344273 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.344613 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.344708 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.347812 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:14.347813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:14.347814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:14.347825 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:14.349201 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.349588 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.349589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.349590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.349667 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.349890 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.349890 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.349890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.349999 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:14.350088 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.350380 (+ 292us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.350479 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:42:14.350703 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.340105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33671 (local address 127.23.197.65:43945)
0805 22:42:14.340267 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:14.340304 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:14.340860 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:14.341178 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:14.341182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:14.341485 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:14.341680 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.341684 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.342785 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.342786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.344192 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.344195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.344749 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.344751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.344936 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.345585 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:14.345597 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:14.347659 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:14.349275 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.349282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.349284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.349517 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.349729 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.349729 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.349730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.349833 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.350055 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:14.350060 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:14.350249 (+ 189us) client_negotiation.cc:769] Sending connection context
0805 22:42:14.350351 (+ 102us) client_negotiation.cc:240] Negotiation successful
0805 22:42:14.350482 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260805 22:42:14.351233 27176 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33671
I20260805 22:42:14.351326 27176 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:14.351528 27176 heartbeater.cc:511] Master 127.23.197.126:33671 requested a full tablet report, sending...
I20260805 22:42:14.351663 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 26984
I20260805 22:42:14.351765 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:14.351974 24341 external_mini_cluster.cc:1428] Setting key 5446c2be7c1d850f9323b54c3e4c880d
I20260805 22:42:14.352151 26875 ts_manager.cc:194] Registered new tserver with Master: 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203)
I20260805 22:42:14.353034 26875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:43945
I20260805 22:42:14.358649 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.352406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33671 (local address 127.23.197.65:50545)
0805 22:42:14.352565 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:14.352578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:14.352642 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:14.352791 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:14.352792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:14.352987 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:14.353050 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.353052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.354028 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.354030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.354572 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.354574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.354952 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.354953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.355028 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.355507 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:14.355512 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:14.355859 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:14.357529 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.357530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.357531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.357719 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.357941 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.357942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.357943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.358010 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.358281 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:14.358282 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:14.358345 (+ 63us) client_negotiation.cc:769] Sending connection context
0805 22:42:14.358427 (+ 82us) client_negotiation.cc:240] Negotiation successful
0805 22:42:14.358544 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260805 22:42:14.358647 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.352537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:50545 (local address 127.23.197.126:33671)
0805 22:42:14.352593 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.352595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.352629 (+ 34us) server_negotiation.cc:407] Connection header received
0805 22:42:14.352696 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.352697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.352705 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.352747 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:14.353132 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.353924 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.354663 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.354861 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.354950 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.355972 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:14.355973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:14.355973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:14.355981 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:14.357426 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.357805 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.357806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.357807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.357868 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.358075 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.358077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.358077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.358203 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:14.358279 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.358468 (+ 189us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.358548 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":192,"server-negotiator.queue_time_us":35}
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:14.383962 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33671
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:14.474586 27189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:14.474815 27189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:14.474843 27189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:14.476001 27189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:14.476046 27189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:14.476068 27189 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260805 22:42:14.476079 27189 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 22:42:14.476097 27189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:14.478395 27189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.197.126:33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:14.478693 27189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:14.478945 27189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:14.484547 27204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:14.484517 27200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:14.484587 27208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:14.484614 27189 server_base.cc:1034] running on GCE node
I20260805 22:42:14.485001 27189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:14.485286 27189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:14.486456 27189 hybrid_clock.cc:648] HybridClock initialized: now 1785969734486444 us; error 31 us; skew 500 ppm
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:14.489231 27189 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:14.489624 27189 webserver.cc:460] Webserver started at http://127.23.197.66:36697/ using document root <none> and password file <none>
I20260805 22:42:14.489830 27189 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:14.489871 27189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:14.489964 27189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:14.491292 27189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "d252e1c10e67449cb4ef81b10ca97c82"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "8b5050490039e02f750bf984de838c07"
server_key_iv: "570fa3ae888e44d496a03dbf37a15c33"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.491632 27189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "d252e1c10e67449cb4ef81b10ca97c82"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "8b5050490039e02f750bf984de838c07"
server_key_iv: "570fa3ae888e44d496a03dbf37a15c33"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.493009 27189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:14.493741 27215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:14.493938 27189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:14.494055 27189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "d252e1c10e67449cb4ef81b10ca97c82"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "8b5050490039e02f750bf984de838c07"
server_key_iv: "570fa3ae888e44d496a03dbf37a15c33"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.494130 27189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:14.521091 27189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:14.523515 27189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:14.523744 27189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:14.524470 27189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:14.524528 27189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:14.524564 27189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:14.524582 27189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:14.533583 27189 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:40883
I20260805 22:42:14.533759 27390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:40883 every 8 connection(s)
I20260805 22:42:14.534018 27189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:14.535971 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.525050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:51137 (local address 127.23.197.126:33671)
0805 22:42:14.525127 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.525130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.525332 (+ 202us) server_negotiation.cc:407] Connection header received
0805 22:42:14.525906 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.525907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.525922 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.526055 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:14.526700 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.527637 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.529135 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.529344 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.529410 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.533005 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:14.533007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:14.533007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:14.533016 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:14.534744 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.535134 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.535135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.535136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.535198 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.535445 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.535445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.535446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.535559 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:14.535617 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.535841 (+ 224us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.535908 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 22:42:14.536170 27228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.525006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33671 (local address 127.23.197.66:51137)
0805 22:42:14.525189 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:14.525232 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:14.525768 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:14.526131 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:14.526135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:14.526380 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:14.526576 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.526580 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.527767 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.527768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.529005 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.529008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.529441 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.529442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.529619 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.530264 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:14.530276 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:14.532876 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:14.534819 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.534824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.534825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.535045 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.535274 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.535275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.535275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.535383 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.535610 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:14.535612 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:14.535735 (+ 123us) client_negotiation.cc:769] Sending connection context
0805 22:42:14.535814 (+ 79us) client_negotiation.cc:240] Negotiation successful
0805 22:42:14.535933 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260805 22:42:14.536783 27392 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33671
I20260805 22:42:14.536897 27392 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:14.537088 27392 heartbeater.cc:511] Master 127.23.197.126:33671 requested a full tablet report, sending...
I20260805 22:42:14.537485 26876 ts_manager.cc:194] Registered new tserver with Master: d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66:40883)
I20260805 22:42:14.538041 26876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:51137
I20260805 22:42:14.541019 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 27189
I20260805 22:42:14.541134 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:14.541393 24341 external_mini_cluster.cc:1428] Setting key a17a7a632a13ca055f21d3aef4a9a62d
I20260805 22:42:14.544266 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.538008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:38571 (local address 127.23.197.126:33671)
0805 22:42:14.538048 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.538050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.538154 (+ 104us) server_negotiation.cc:407] Connection header received
0805 22:42:14.538222 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.538223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.538234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.538298 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:14.538685 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.539138 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.539845 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.540017 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.540069 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.541294 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:14.541295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:14.541295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:14.541306 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:14.542937 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.543307 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.543308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.543308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.543374 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.543579 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.543580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.543581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.543766 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:14.543851 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.544065 (+ 214us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.544194 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:42:14.544283 27401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.537938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33671 (local address 127.23.197.66:38571)
0805 22:42:14.538075 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:14.538086 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:14.538167 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:14.538317 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:14.538319 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:14.538540 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:14.538602 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.538604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.539254 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.539280 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.539770 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.539772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.540119 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.540120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.540188 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.540624 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:14.540628 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:14.541162 (+ 534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:14.543028 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.543029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.543030 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.543219 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.543448 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.543448 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.543449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.543516 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.543853 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:14.543855 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:14.543909 (+ 54us) client_negotiation.cc:769] Sending connection context
0805 22:42:14.543998 (+ 89us) client_negotiation.cc:240] Negotiation successful
0805 22:42:14.544144 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:14.569504 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33671
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:14.659500 27406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:14.659709 27406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:14.659734 27406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:14.660907 27406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:14.660948 27406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:14.660969 27406 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260805 22:42:14.660981 27406 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 22:42:14.660998 27406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:14.662443 27406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.197.126:33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:14.662637 27406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:14.662833 27406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:14.668207 27413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:14.668211 27416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:14.668231 27419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:14.668505 27406 server_base.cc:1034] running on GCE node
I20260805 22:42:14.668988 27406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:14.669269 27406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:14.670472 27406 hybrid_clock.cc:648] HybridClock initialized: now 1785969734670457 us; error 27 us; skew 500 ppm
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:14.673240 27406 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:14.673779 27406 webserver.cc:460] Webserver started at http://127.23.197.67:37747/ using document root <none> and password file <none>
I20260805 22:42:14.674096 27406 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:14.674175 27406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:14.674348 27406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:14.675511 27406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "39f722e7986f4388a43a559e2564c98e"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "2e79d5e0d498ff4e00fb022ed4465729"
server_key_iv: "e62f59d177ce3786e15a22342170b042"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.675922 27406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "39f722e7986f4388a43a559e2564c98e"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "2e79d5e0d498ff4e00fb022ed4465729"
server_key_iv: "e62f59d177ce3786e15a22342170b042"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.677560 27406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:14.678406 27431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:14.678604 27406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:14.678709 27406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "39f722e7986f4388a43a559e2564c98e"
format_stamp: "Formatted at 2026-08-05 22:42:14 on dist-test-slave-g170"
server_key: "2e79d5e0d498ff4e00fb022ed4465729"
server_key_iv: "e62f59d177ce3786e15a22342170b042"
server_key_version: "encryptionkey@0"
I20260805 22:42:14.678799 27406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:14.688278 27406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:14.690706 27406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:14.690918 27406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:14.691505 27406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:14.691567 27406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:14.691612 27406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:14.691644 27406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:14.699290 27406 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:39253
I20260805 22:42:14.699703 27406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:14.699949 27589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:39253 every 8 connection(s)
I20260805 22:42:14.702026 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.691948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:40403 (local address 127.23.197.126:33671)
0805 22:42:14.692012 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.692014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.692216 (+ 202us) server_negotiation.cc:407] Connection header received
0805 22:42:14.692813 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.692815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.692829 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.692894 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:14.693544 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.694069 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.695193 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.695405 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.695496 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.698827 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:14.698829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:14.698829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:14.698837 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:14.700684 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.701186 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.701187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.701187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.701251 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.701479 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.701480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.701480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.701596 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:14.701653 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.701888 (+ 235us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.701957 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 22:42:14.702152 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.691929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33671 (local address 127.23.197.67:40403)
0805 22:42:14.692095 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:14.692131 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:14.692675 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:14.692966 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:14.692970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:14.693238 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:14.693438 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.693442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.694175 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.694176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.695069 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.695072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.695509 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.695510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.695704 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.696322 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:14.696333 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:14.698722 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:14.700816 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.700823 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.700825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.701112 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.701320 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.701320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.701320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.701419 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.701653 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:14.701657 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:14.701769 (+ 112us) client_negotiation.cc:769] Sending connection context
0805 22:42:14.701849 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 22:42:14.701969 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260805 22:42:14.702672 27591 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33671
I20260805 22:42:14.702821 27591 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:14.703042 27591 heartbeater.cc:511] Master 127.23.197.126:33671 requested a full tablet report, sending...
I20260805 22:42:14.703470 26875 ts_manager.cc:194] Registered new tserver with Master: 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253)
I20260805 22:42:14.704056 26875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:40403
I20260805 22:42:14.706192 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 27406
I20260805 22:42:14.706282 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:14.706528 24341 external_mini_cluster.cc:1428] Setting key 0453ffcafeb2d5642ad12804fe6c7d03
I20260805 22:42:14.707417 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:42:14.710353 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.703907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:44123 (local address 127.23.197.126:33671)
0805 22:42:14.704225 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.704227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.704235 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:14.704256 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.704257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.704268 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.704323 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:14.704796 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.705274 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.706047 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.706234 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.706326 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.707511 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:14.707513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:14.707514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:14.707521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:14.709011 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.709462 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.709463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.709464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.709515 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.709738 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.709739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.709739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.709841 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:14.709894 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.710126 (+ 232us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.710230 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294}
I20260805 22:42:14.710363 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.703809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33671 (local address 127.23.197.67:44123)
0805 22:42:14.703954 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:14.703966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:14.704024 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:14.704347 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:14.704349 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:14.704579 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:14.704656 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.704658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.705372 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.705373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.705937 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.705939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.706346 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.706347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.706436 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.706933 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:14.706940 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:14.707404 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:14.709113 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.709114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.709115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.709387 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.709585 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:14.709586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:14.709587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:14.709668 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:14.709910 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:14.709911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:14.709968 (+ 57us) client_negotiation.cc:769] Sending connection context
0805 22:42:14.710088 (+ 120us) client_negotiation.cc:240] Negotiation successful
0805 22:42:14.710240 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 22:42:14 dist-test-slave-g170 krb5kdc[26813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:14.734584 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.726820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50376 (local address 127.23.197.126:33671)
0805 22:42:14.726892 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.726894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.726901 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:42:14.726926 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.726927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.726935 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.727009 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:14.727705 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.728231 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.729122 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.729316 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.729425 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.731842 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:14.731843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:14.731844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:14.731852 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:14.733461 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.733876 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.733877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.733878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.733934 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:14.734122 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:14.734123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:14.734123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:14.734239 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:14.734319 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.734450 (+ 131us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.734515 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 22:42:14.736603 26875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:14.738668 26875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:14.745559 27613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.741296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40883 (local address 127.0.0.1:39180)
0805 22:42:14.741420 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:14.741461 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:14.741555 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:14.742036 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:14.742040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:14.742043 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:14.742173 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.742177 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.743257 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.743258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.744270 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.744273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.745207 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.745208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.745311 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.745329 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 22:42:14.745412 (+ 83us) client_negotiation.cc:240] Negotiation successful
0805 22:42:14.745471 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":76,"threads_started":1}
I20260805 22:42:14.745579 27621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.741597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.23.197.66:40883)
0805 22:42:14.741841 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.741844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.741853 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:14.741902 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.741904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.741922 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.742005 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:14.742252 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.743170 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.744373 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.745143 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.745211 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.745281 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.745437 (+ 156us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.745479 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":145,"threads_started":1}
I20260805 22:42:14.746315 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.741378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:39253 (local address 127.0.0.1:53086)
0805 22:42:14.741672 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:14.741680 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:14.741719 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:14.742568 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:14.742569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:14.742571 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:14.742607 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.742609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.743946 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.743947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.745017 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.745021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.746076 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.746078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.746140 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.746145 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:14.746203 (+ 58us) client_negotiation.cc:240] Negotiation successful
0805 22:42:14.746245 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260805 22:42:14.746529 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.741671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47890 (local address 127.23.197.65:41203)
0805 22:42:14.741878 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.741884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.741899 (+ 15us) server_negotiation.cc:407] Connection header received
0805 22:42:14.741943 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.741945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.741969 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.742260 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:14.742490 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.743777 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.745074 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.745953 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.746028 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.746088 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.746344 (+ 256us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.746415 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":87,"threads_started":1}
I20260805 22:42:14.746661 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.741960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53086 (local address 127.23.197.67:39253)
0805 22:42:14.742399 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:14.742404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:14.742413 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:14.742453 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:14.742455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:14.742475 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:14.742569 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:14.742669 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.743838 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.745121 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.745907 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.745983 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.746040 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:14.746249 (+ 209us) server_negotiation.cc:299] Negotiation successful
0805 22:42:14.746307 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":112,"threads_started":1}
I20260805 22:42:14.746542 27290 tablet_service.cc:1467] Processing CreateTablet for tablet c21ed349b38d4f4faf12c4041f6eb1aa (DEFAULT_TABLE table=test-table [id=79eacc2501e541169acaa5f137d5fe5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:14.746917 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:14.741378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:41203 (local address 127.0.0.1:47890)
0805 22:42:14.741549 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:14.741558 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:14.741599 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:14.742297 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:14.742298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:14.742300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:14.742361 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.742364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.743886 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.743887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.744982 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:14.744985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:14.746126 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:14.746127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:14.746233 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:14.746238 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:14.746776 (+ 538us) client_negotiation.cc:240] Negotiation successful
0805 22:42:14.746833 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":115,"threads_started":1}
I20260805 22:42:14.746955 27290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21ed349b38d4f4faf12c4041f6eb1aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:14.747586 27082 tablet_service.cc:1467] Processing CreateTablet for tablet c21ed349b38d4f4faf12c4041f6eb1aa (DEFAULT_TABLE table=test-table [id=79eacc2501e541169acaa5f137d5fe5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:14.748021 27082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21ed349b38d4f4faf12c4041f6eb1aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:14.750591 27624 tablet_bootstrap.cc:492] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: Bootstrap starting.
I20260805 22:42:14.750816 27501 tablet_service.cc:1467] Processing CreateTablet for tablet c21ed349b38d4f4faf12c4041f6eb1aa (DEFAULT_TABLE table=test-table [id=79eacc2501e541169acaa5f137d5fe5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:14.751169 27501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21ed349b38d4f4faf12c4041f6eb1aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:14.751339 27624 tablet_bootstrap.cc:654] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:14.751704 27624 log.cc:824] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:14.751889 27625 tablet_bootstrap.cc:492] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: Bootstrap starting.
I20260805 22:42:14.752622 27624 tablet_bootstrap.cc:492] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: No bootstrap required, opened a new log
I20260805 22:42:14.752702 27624 ts_tablet_manager.cc:1397] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:42:14.752722 27625 tablet_bootstrap.cc:654] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:14.753078 27625 log.cc:824] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:14.754624 27628 tablet_bootstrap.cc:492] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: Bootstrap starting.
I20260805 22:42:14.754896 27624 raft_consensus.cc:348] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:14.755023 27624 raft_consensus.cc:374] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:14.755044 27624 raft_consensus.cc:729] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d252e1c10e67449cb4ef81b10ca97c82, State: Initialized, Role: FOLLOWER
I20260805 22:42:14.755160 27624 consensus_queue.cc:260] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:14.755448 27624 ts_tablet_manager.cc:1428] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:42:14.755518 27392 heartbeater.cc:503] Master 127.23.197.126:33671 was elected leader, sending a full tablet report...
I20260805 22:42:14.755620 27628 tablet_bootstrap.cc:654] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:14.756135 27625 tablet_bootstrap.cc:492] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: No bootstrap required, opened a new log
I20260805 22:42:14.756212 27625 ts_tablet_manager.cc:1397] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260805 22:42:14.758096 27625 raft_consensus.cc:348] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:14.758229 27625 raft_consensus.cc:374] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:14.758250 27625 raft_consensus.cc:729] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736645fea67f4798ae7f44251332f7d6, State: Initialized, Role: FOLLOWER
I20260805 22:42:14.758380 27625 consensus_queue.cc:260] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:14.758610 27625 ts_tablet_manager.cc:1428] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260805 22:42:14.758687 27176 heartbeater.cc:503] Master 127.23.197.126:33671 was elected leader, sending a full tablet report...
I20260805 22:42:14.758983 27628 log.cc:824] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:14.759828 27628 tablet_bootstrap.cc:492] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: No bootstrap required, opened a new log
I20260805 22:42:14.759910 27628 ts_tablet_manager.cc:1397] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260805 22:42:14.761389 27628 raft_consensus.cc:348] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:14.761538 27628 raft_consensus.cc:374] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:14.761592 27628 raft_consensus.cc:729] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f722e7986f4388a43a559e2564c98e, State: Initialized, Role: FOLLOWER
I20260805 22:42:14.761721 27628 consensus_queue.cc:260] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:14.761971 27628 ts_tablet_manager.cc:1428] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:14.762023 27591 heartbeater.cc:503] Master 127.23.197.126:33671 was elected leader, sending a full tablet report...
W20260805 22:42:14.784709 27393 tablet.cc:2369] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:42:14.848460 27177 tablet.cc:2369] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:42:14.950543 27592 tablet.cc:2369] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:42:15.008358 27629 raft_consensus.cc:482] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:15.008509 27629 raft_consensus.cc:504] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.009037 27629 leader_election.cc:288] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203), 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253)
I20260805 22:42:15.013700 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.009459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:52993 (local address 127.23.197.65:41203)
0805 22:42:15.009516 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:15.009518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:15.009579 (+ 61us) server_negotiation.cc:407] Connection header received
0805 22:42:15.009707 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:15.009708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:15.009717 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:15.009798 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:15.010087 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.010882 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.012570 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.013220 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.013313 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.013324 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:15.013571 (+ 247us) server_negotiation.cc:299] Negotiation successful
0805 22:42:15.013626 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260805 22:42:15.013720 27228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.009354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:39253 (local address 127.23.197.66:49151)
0805 22:42:15.009427 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:15.009441 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:15.009497 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:15.009830 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:15.009831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:15.009833 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:15.009960 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.009963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.011102 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.011103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.012433 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.012437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.013350 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.013352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.013464 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.013469 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:15.013549 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 22:42:15.013621 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":106}
I20260805 22:42:15.013763 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.009492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:49151 (local address 127.23.197.67:39253)
0805 22:42:15.009578 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:15.009581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:15.009590 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:15.009625 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:15.009626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:15.009645 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:15.009800 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:15.010087 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.010916 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.012563 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.013220 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.013316 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.013324 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:15.013570 (+ 246us) server_negotiation.cc:299] Negotiation successful
0805 22:42:15.013674 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 22:42:15.013981 27635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.009355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:41203 (local address 127.23.197.66:52993)
0805 22:42:15.009506 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:15.009515 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:15.009629 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:15.009829 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:15.009831 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:15.009833 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:15.009970 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.009972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.011102 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.011103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.012433 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.012436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.013350 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.013352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.013465 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.013469 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:15.013549 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 22:42:15.013673 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":92,"thread_start_us":76,"threads_started":1}
I20260805 22:42:15.014245 27526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c21ed349b38d4f4faf12c4041f6eb1aa" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f722e7986f4388a43a559e2564c98e" is_pre_election: true
I20260805 22:42:15.014474 27526 raft_consensus.cc:2454] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 0.
I20260805 22:42:15.014433 27108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c21ed349b38d4f4faf12c4041f6eb1aa" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736645fea67f4798ae7f44251332f7d6" is_pre_election: true
I20260805 22:42:15.014605 27108 raft_consensus.cc:2454] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 0.
I20260805 22:42:15.014791 27217 leader_election.cc:302] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f722e7986f4388a43a559e2564c98e, d252e1c10e67449cb4ef81b10ca97c82; no voters:
I20260805 22:42:15.014993 27629 raft_consensus.cc:2781] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:15.015038 27629 raft_consensus.cc:482] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:15.015056 27629 raft_consensus.cc:3037] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:15.015985 27629 raft_consensus.cc:504] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.016224 27629 leader_election.cc:288] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 election: Requested vote from peers 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203), 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253)
I20260805 22:42:15.016330 27108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c21ed349b38d4f4faf12c4041f6eb1aa" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736645fea67f4798ae7f44251332f7d6"
I20260805 22:42:15.016386 27526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c21ed349b38d4f4faf12c4041f6eb1aa" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f722e7986f4388a43a559e2564c98e"
I20260805 22:42:15.016422 27108 raft_consensus.cc:3037] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:15.016481 27526 raft_consensus.cc:3037] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:15.017469 27526 raft_consensus.cc:2454] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 1.
I20260805 22:42:15.017668 27217 leader_election.cc:302] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f722e7986f4388a43a559e2564c98e, d252e1c10e67449cb4ef81b10ca97c82; no voters:
I20260805 22:42:15.017688 27108 raft_consensus.cc:2454] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 1.
I20260805 22:42:15.019654 27629 raft_consensus.cc:2781] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:15.019857 27629 raft_consensus.cc:686] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 1 LEADER]: Becoming Leader. State: Replica: d252e1c10e67449cb4ef81b10ca97c82, State: Running, Role: LEADER
I20260805 22:42:15.019981 27629 consensus_queue.cc:237] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.021016 26875 catalog_manager.cc:5495] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 reported cstate change: term changed from 0 to 1, leader changed from <none> to d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } health_report { overall_health: UNKNOWN } } }
Aug 05 22:42:15 dist-test-slave-g170 krb5kdc[26813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969734, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:15.053499 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.045719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.23.197.126:33671)
0805 22:42:15.045814 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:15.045817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:15.045935 (+ 118us) server_negotiation.cc:407] Connection header received
0805 22:42:15.046042 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:15.046043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:15.046053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:15.046119 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:15.046706 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.047203 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.048027 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.048211 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.048273 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.050697 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:15.050699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:15.050699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:15.050707 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:15.052278 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:15.052731 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:15.052732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:15.052732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:15.052873 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:15.053074 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:15.053075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:15.053075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:15.053184 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:15.053234 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:15.053364 (+ 130us) server_negotiation.cc:299] Negotiation successful
0805 22:42:15.053433 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 22:42:15.053720 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.045536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33671 (local address 127.0.0.1:50382)
0805 22:42:15.045855 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:15.045865 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:15.045959 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:15.046142 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:15.046144 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:15.046496 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:15.046608 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.046612 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.047282 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.047283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.047950 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.047953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.048310 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.048311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.048364 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.049080 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:15.049085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:15.050600 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:15.052381 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:15.052389 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:15.052392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:15.052658 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:15.052941 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:15.052942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:15.052942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:15.053008 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:15.053241 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:15.053249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:15.053293 (+ 44us) client_negotiation.cc:769] Sending connection context
0805 22:42:15.053349 (+ 56us) client_negotiation.cc:240] Negotiation successful
0805 22:42:15.053496 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":45,"thread_start_us":205,"threads_started":1}
I20260805 22:42:15.056052 26875 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:50382:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260805 22:42:15.056283 26875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:15.057964 27082 tablet_service.cc:1467] Processing CreateTablet for tablet 4dd562b5fbe34e3184d4a55e69b72c70 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83951d72f9a249058055177d9ba6dd25]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 22:42:15.058060 27501 tablet_service.cc:1467] Processing CreateTablet for tablet 4dd562b5fbe34e3184d4a55e69b72c70 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83951d72f9a249058055177d9ba6dd25]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 22:42:15.058034 27290 tablet_service.cc:1467] Processing CreateTablet for tablet 4dd562b5fbe34e3184d4a55e69b72c70 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83951d72f9a249058055177d9ba6dd25]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 22:42:15.058166 27082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dd562b5fbe34e3184d4a55e69b72c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:15.058214 27501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dd562b5fbe34e3184d4a55e69b72c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:15.058259 27290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dd562b5fbe34e3184d4a55e69b72c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:15.059549 27624 tablet_bootstrap.cc:492] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: Bootstrap starting.
I20260805 22:42:15.059715 27628 tablet_bootstrap.cc:492] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: Bootstrap starting.
I20260805 22:42:15.060281 27624 tablet_bootstrap.cc:654] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:15.060290 27628 tablet_bootstrap.cc:654] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:15.060681 27625 tablet_bootstrap.cc:492] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: Bootstrap starting.
I20260805 22:42:15.061093 27628 tablet_bootstrap.cc:492] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: No bootstrap required, opened a new log
I20260805 22:42:15.061133 27625 tablet_bootstrap.cc:654] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:15.061161 27628 ts_tablet_manager.cc:1397] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:15.061333 27628 raft_consensus.cc:348] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.061437 27628 raft_consensus.cc:374] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:15.061463 27628 raft_consensus.cc:729] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f722e7986f4388a43a559e2564c98e, State: Initialized, Role: FOLLOWER
I20260805 22:42:15.061507 27628 consensus_queue.cc:260] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.061619 27628 ts_tablet_manager.cc:1428] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:15.061676 27633 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.061785 27633 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:15.061866 27624 tablet_bootstrap.cc:492] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: No bootstrap required, opened a new log
I20260805 22:42:15.061904 27624 ts_tablet_manager.cc:1397] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260805 22:42:15.061966 27625 tablet_bootstrap.cc:492] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: No bootstrap required, opened a new log
I20260805 22:42:15.062034 27625 ts_tablet_manager.cc:1397] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:15.062053 27624 raft_consensus.cc:348] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.062119 27624 raft_consensus.cc:374] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:15.062136 27624 raft_consensus.cc:729] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d252e1c10e67449cb4ef81b10ca97c82, State: Initialized, Role: FOLLOWER
I20260805 22:42:15.062211 27624 consensus_queue.cc:260] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.062209 27625 raft_consensus.cc:348] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.062263 27625 raft_consensus.cc:374] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:15.062276 27625 raft_consensus.cc:729] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736645fea67f4798ae7f44251332f7d6, State: Initialized, Role: FOLLOWER
I20260805 22:42:15.062301 27624 ts_tablet_manager.cc:1428] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:15.062336 27625 consensus_queue.cc:260] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.062337 27629 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.062402 27629 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:15.062448 27625 ts_tablet_manager.cc:1428] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:15.062477 27631 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.062559 27631 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:15.100732 27629 raft_consensus.cc:482] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:15.100895 27629 raft_consensus.cc:504] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.101104 27629 leader_election.cc:288] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203), 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253)
I20260805 22:42:15.101341 27108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dd562b5fbe34e3184d4a55e69b72c70" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736645fea67f4798ae7f44251332f7d6" is_pre_election: true
I20260805 22:42:15.101341 27526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dd562b5fbe34e3184d4a55e69b72c70" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f722e7986f4388a43a559e2564c98e" is_pre_election: true
I20260805 22:42:15.101446 27526 raft_consensus.cc:2454] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 0.
I20260805 22:42:15.101446 27108 raft_consensus.cc:2454] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 0.
I20260805 22:42:15.101585 27217 leader_election.cc:302] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f722e7986f4388a43a559e2564c98e, d252e1c10e67449cb4ef81b10ca97c82; no voters:
I20260805 22:42:15.101712 27629 raft_consensus.cc:2781] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:15.101763 27629 raft_consensus.cc:482] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:15.101783 27629 raft_consensus.cc:3037] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:15.102511 27629 raft_consensus.cc:504] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.102655 27629 leader_election.cc:288] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 election: Requested vote from peers 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203), 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253)
I20260805 22:42:15.102830 27108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dd562b5fbe34e3184d4a55e69b72c70" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736645fea67f4798ae7f44251332f7d6"
I20260805 22:42:15.102918 27108 raft_consensus.cc:3037] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:15.102869 27526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dd562b5fbe34e3184d4a55e69b72c70" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f722e7986f4388a43a559e2564c98e"
I20260805 22:42:15.102969 27526 raft_consensus.cc:3037] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:15.103673 27526 raft_consensus.cc:2454] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 1.
I20260805 22:42:15.103708 27108 raft_consensus.cc:2454] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 1.
I20260805 22:42:15.103896 27218 leader_election.cc:302] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 736645fea67f4798ae7f44251332f7d6, d252e1c10e67449cb4ef81b10ca97c82; no voters:
I20260805 22:42:15.104022 27629 raft_consensus.cc:2781] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:15.104105 27629 raft_consensus.cc:686] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 LEADER]: Becoming Leader. State: Replica: d252e1c10e67449cb4ef81b10ca97c82, State: Running, Role: LEADER
I20260805 22:42:15.104162 27629 consensus_queue.cc:237] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.104447 27637 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: TxnStatusTablet state changed. Reason: New leader d252e1c10e67449cb4ef81b10ca97c82. Latest consensus state: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.104552 27637 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: This TxnStatusTablet replica's current role is: LEADER
I20260805 22:42:15.104856 27657 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260805 22:42:15.104874 26876 catalog_manager.cc:5495] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 reported cstate change: term changed from 0 to 1, leader changed from <none> to d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:15.105002 27657 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260805 22:42:15.111810 27621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.109142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.23.197.66:40883)
0805 22:42:15.109204 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:15.109206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:15.109216 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:15.109310 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:15.109311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:15.109328 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:15.109404 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:15.109553 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.110031 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.110825 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.111102 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.111178 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.111447 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:15.111565 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:15.111655 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:15.111716 (+ 61us) server_negotiation.cc:299] Negotiation successful
0805 22:42:15.111739 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 22:42:15.111830 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.109054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40883 (local address 127.0.0.1:39196)
0805 22:42:15.109133 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:15.109145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:15.109223 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:15.109425 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:15.109427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:15.109429 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:15.109477 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.109480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.110112 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.110114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.110696 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.110698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.111191 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.111193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.111261 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.111284 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:15.111648 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:15.111659 (+ 11us) client_negotiation.cc:769] Sending connection context
0805 22:42:15.111701 (+ 42us) client_negotiation.cc:240] Negotiation successful
0805 22:42:15.111761 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260805 22:42:15.113832 27526 raft_consensus.cc:1261] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Refusing update from remote peer d252e1c10e67449cb4ef81b10ca97c82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:15.113854 27108 raft_consensus.cc:1261] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Refusing update from remote peer d252e1c10e67449cb4ef81b10ca97c82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:15.114256 27637 consensus_queue.cc:1035] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:15.114406 27629 consensus_queue.cc:1035] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:15.115646 27633 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: TxnStatusTablet state changed. Reason: New leader d252e1c10e67449cb4ef81b10ca97c82. Latest consensus state: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.115760 27633 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:15.115635 27631 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: TxnStatusTablet state changed. Reason: New leader d252e1c10e67449cb4ef81b10ca97c82. Latest consensus state: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.116371 27631 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:15.116969 27637 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.117054 27637 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: This TxnStatusTablet replica's current role is: LEADER
I20260805 22:42:15.117455 27637 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.117553 27637 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P d252e1c10e67449cb4ef81b10ca97c82: This TxnStatusTablet replica's current role is: LEADER
I20260805 22:42:15.117671 27633 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.117767 27633 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P 39f722e7986f4388a43a559e2564c98e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:15.118008 27665 mvcc.cc:205] Tried to move back new op lower bound from 7315332035023564800 to 7315332034987073536. Current Snapshot: MvccSnapshot[applied={T|T < 7315332035023564800}]
I20260805 22:42:15.118088 27631 tablet_replica.cc:439] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } } }
I20260805 22:42:15.118177 27631 tablet_replica.cc:442] T 4dd562b5fbe34e3184d4a55e69b72c70 P 736645fea67f4798ae7f44251332f7d6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:15.118782 27669 mvcc.cc:205] Tried to move back new op lower bound from 7315332035023564800 to 7315332034987073536. Current Snapshot: MvccSnapshot[applied={T|T < 7315332035023564800}]
I20260805 22:42:15.122720 27621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.120071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.23.197.66:40883)
0805 22:42:15.120113 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:15.120115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:15.120167 (+ 52us) server_negotiation.cc:407] Connection header received
0805 22:42:15.120239 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:15.120240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:15.120248 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:15.120287 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:15.120429 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.120967 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.122092 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.122280 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.122335 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.122509 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:15.122536 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:15.122573 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:15.122655 (+ 82us) server_negotiation.cc:299] Negotiation successful
0805 22:42:15.122671 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:42:15.127902 27621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.125325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:58333 (local address 127.23.197.66:40883)
0805 22:42:15.125440 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:15.125442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:15.125512 (+ 70us) server_negotiation.cc:407] Connection header received
0805 22:42:15.125571 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:15.125571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:15.125581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:15.125633 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:15.125778 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.126291 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.127156 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.127346 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.127413 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.127598 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:15.127627 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:15.127698 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:15.127763 (+ 65us) server_negotiation.cc:299] Negotiation successful
0805 22:42:15.127782 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260805 22:42:15.128113 27684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:15.125269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40883 (local address 127.23.197.66:58333)
0805 22:42:15.125444 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:15.125455 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:15.125515 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:15.125637 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:15.125637 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:15.125639 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:15.125706 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.125711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.126372 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.126373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.127067 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:15.127069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:15.127441 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:15.127442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:15.127497 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:15.127510 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:15.127697 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:15.127698 (+ 1us) client_negotiation.cc:769] Sending connection context
0805 22:42:15.127763 (+ 65us) client_negotiation.cc:240] Negotiation successful
0805 22:42:15.127804 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":106,"threads_started":1}
I20260805 22:42:15.130527 27526 raft_consensus.cc:1261] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Refusing update from remote peer d252e1c10e67449cb4ef81b10ca97c82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:15.130565 27108 raft_consensus.cc:1261] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Refusing update from remote peer d252e1c10e67449cb4ef81b10ca97c82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:15.130786 27685 consensus_queue.cc:1035] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:15.130867 27629 consensus_queue.cc:1035] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:15.146193 26876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:15.146320 26876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:15.147183 26876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: Sending DeleteTablet for 3 replicas of tablet c21ed349b38d4f4faf12c4041f6eb1aa
I20260805 22:42:15.147585 27082 tablet_service.cc:1514] Processing DeleteTablet for tablet c21ed349b38d4f4faf12c4041f6eb1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:15 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:47890
I20260805 22:42:15.147673 27290 tablet_service.cc:1514] Processing DeleteTablet for tablet c21ed349b38d4f4faf12c4041f6eb1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:15 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:39180
I20260805 22:42:15.147682 27501 tablet_service.cc:1514] Processing DeleteTablet for tablet c21ed349b38d4f4faf12c4041f6eb1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:15 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:53086
I20260805 22:42:15.147982 27699 tablet_replica.cc:330] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: stopping tablet replica
I20260805 22:42:15.148109 27699 raft_consensus.cc:2229] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:15.148172 27699 raft_consensus.cc:2258] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:15.148119 26876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:15.148278 26876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:15.148670 27699 ts_tablet_manager.cc:1905] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:15.149595 27290 tablet_service.cc:1467] Processing CreateTablet for tablet 24b3ade6af0b41a491d8b0f31b2bbf74 (DEFAULT_TABLE table=test-table [id=94bec2f51d39426fb258ea7771671468]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:15.149693 27697 tablet_replica.cc:330] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: stopping tablet replica
I20260805 22:42:15.149748 27290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24b3ade6af0b41a491d8b0f31b2bbf74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:15.149819 27697 raft_consensus.cc:2229] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:15.149878 27697 raft_consensus.cc:2258] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:15.150250 27697 ts_tablet_manager.cc:1905] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:15.151113 27501 tablet_service.cc:1467] Processing CreateTablet for tablet 24b3ade6af0b41a491d8b0f31b2bbf74 (DEFAULT_TABLE table=test-table [id=94bec2f51d39426fb258ea7771671468]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:15.151229 27501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24b3ade6af0b41a491d8b0f31b2bbf74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:15.152164 27697 ts_tablet_manager.cc:1918] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:15.152271 27697 log.cc:1196] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/c21ed349b38d4f4faf12c4041f6eb1aa
I20260805 22:42:15.152604 27697 ts_tablet_manager.cc:1939] T c21ed349b38d4f4faf12c4041f6eb1aa P 736645fea67f4798ae7f44251332f7d6: Deleting consensus metadata
I20260805 22:42:15.152717 27624 tablet_bootstrap.cc:492] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: Bootstrap starting.
I20260805 22:42:15.153168 27624 tablet_bootstrap.cc:654] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:15.153558 27628 tablet_bootstrap.cc:492] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: Bootstrap starting.
I20260805 22:42:15.153966 27624 tablet_bootstrap.cc:492] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: No bootstrap required, opened a new log
I20260805 22:42:15.153978 27628 tablet_bootstrap.cc:654] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:15.154039 27624 ts_tablet_manager.cc:1397] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:15.154193 27624 raft_consensus.cc:348] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.154286 27624 raft_consensus.cc:374] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:15.154306 27624 raft_consensus.cc:729] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d252e1c10e67449cb4ef81b10ca97c82, State: Initialized, Role: FOLLOWER
I20260805 22:42:15.154362 27624 consensus_queue.cc:260] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.154448 27624 ts_tablet_manager.cc:1428] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:15.154541 26857 catalog_manager.cc:4841] TS 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203): tablet c21ed349b38d4f4faf12c4041f6eb1aa (table test-table [id=79eacc2501e541169acaa5f137d5fe5a]) successfully deleted
I20260805 22:42:15.154779 27082 tablet_service.cc:1467] Processing CreateTablet for tablet 24b3ade6af0b41a491d8b0f31b2bbf74 (DEFAULT_TABLE table=test-table [id=94bec2f51d39426fb258ea7771671468]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:15.154897 27082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24b3ade6af0b41a491d8b0f31b2bbf74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:15.156131 27625 tablet_bootstrap.cc:492] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: Bootstrap starting.
I20260805 22:42:15.156347 27702 tablet_replica.cc:330] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: stopping tablet replica
I20260805 22:42:15.156476 27702 raft_consensus.cc:2229] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:15.156600 27702 raft_consensus.cc:2258] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:15.156662 27625 tablet_bootstrap.cc:654] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:15.157332 27625 tablet_bootstrap.cc:492] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: No bootstrap required, opened a new log
I20260805 22:42:15.157400 27625 ts_tablet_manager.cc:1397] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:15.157554 27625 raft_consensus.cc:348] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.157645 27625 raft_consensus.cc:374] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:15.157672 27625 raft_consensus.cc:729] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736645fea67f4798ae7f44251332f7d6, State: Initialized, Role: FOLLOWER
I20260805 22:42:15.157716 27625 consensus_queue.cc:260] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.157848 27625 ts_tablet_manager.cc:1428] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:15.158087 27699 ts_tablet_manager.cc:1918] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:15.158172 27699 log.cc:1196] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/c21ed349b38d4f4faf12c4041f6eb1aa
I20260805 22:42:15.158458 27699 ts_tablet_manager.cc:1939] T c21ed349b38d4f4faf12c4041f6eb1aa P 39f722e7986f4388a43a559e2564c98e: Deleting consensus metadata
I20260805 22:42:15.158950 26856 catalog_manager.cc:4841] TS 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253): tablet c21ed349b38d4f4faf12c4041f6eb1aa (table test-table [id=79eacc2501e541169acaa5f137d5fe5a]) successfully deleted
I20260805 22:42:15.159307 27702 ts_tablet_manager.cc:1905] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:15.161171 27628 tablet_bootstrap.cc:492] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: No bootstrap required, opened a new log
I20260805 22:42:15.161190 27702 ts_tablet_manager.cc:1918] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:15.161249 27628 ts_tablet_manager.cc:1397] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260805 22:42:15.161262 27702 log.cc:1196] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/c21ed349b38d4f4faf12c4041f6eb1aa
I20260805 22:42:15.161448 27628 raft_consensus.cc:348] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.161532 27628 raft_consensus.cc:374] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:15.161525 27702 ts_tablet_manager.cc:1939] T c21ed349b38d4f4faf12c4041f6eb1aa P d252e1c10e67449cb4ef81b10ca97c82: Deleting consensus metadata
I20260805 22:42:15.161563 27628 raft_consensus.cc:729] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f722e7986f4388a43a559e2564c98e, State: Initialized, Role: FOLLOWER
I20260805 22:42:15.161607 27628 consensus_queue.cc:260] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.161707 27628 ts_tablet_manager.cc:1428] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:15.161981 26859 catalog_manager.cc:4841] TS d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66:40883): tablet c21ed349b38d4f4faf12c4041f6eb1aa (table test-table [id=79eacc2501e541169acaa5f137d5fe5a]) successfully deleted
I20260805 22:42:15.277536 27693 raft_consensus.cc:482] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:15.277668 27693 raft_consensus.cc:504] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.277868 27693 leader_election.cc:288] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203), 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253)
I20260805 22:42:15.278147 27108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24b3ade6af0b41a491d8b0f31b2bbf74" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736645fea67f4798ae7f44251332f7d6" is_pre_election: true
I20260805 22:42:15.278160 27522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24b3ade6af0b41a491d8b0f31b2bbf74" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f722e7986f4388a43a559e2564c98e" is_pre_election: true
I20260805 22:42:15.278285 27522 raft_consensus.cc:2454] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 0.
I20260805 22:42:15.278287 27108 raft_consensus.cc:2454] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 0.
I20260805 22:42:15.278517 27217 leader_election.cc:302] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f722e7986f4388a43a559e2564c98e, d252e1c10e67449cb4ef81b10ca97c82; no voters:
I20260805 22:42:15.278625 27693 raft_consensus.cc:2781] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:15.278673 27693 raft_consensus.cc:482] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:15.278700 27693 raft_consensus.cc:3037] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:15.279517 27693 raft_consensus.cc:504] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.279771 27693 leader_election.cc:288] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 election: Requested vote from peers 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203), 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253)
I20260805 22:42:15.279886 27108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24b3ade6af0b41a491d8b0f31b2bbf74" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736645fea67f4798ae7f44251332f7d6"
I20260805 22:42:15.279987 27108 raft_consensus.cc:3037] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:15.279975 27522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24b3ade6af0b41a491d8b0f31b2bbf74" candidate_uuid: "d252e1c10e67449cb4ef81b10ca97c82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f722e7986f4388a43a559e2564c98e"
I20260805 22:42:15.280054 27522 raft_consensus.cc:3037] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:15.280925 27522 raft_consensus.cc:2454] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 1.
I20260805 22:42:15.280925 27108 raft_consensus.cc:2454] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d252e1c10e67449cb4ef81b10ca97c82 in term 1.
I20260805 22:42:15.281116 27217 leader_election.cc:302] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f722e7986f4388a43a559e2564c98e, d252e1c10e67449cb4ef81b10ca97c82; no voters:
I20260805 22:42:15.281234 27693 raft_consensus.cc:2781] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:15.281291 27693 raft_consensus.cc:686] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 LEADER]: Becoming Leader. State: Replica: d252e1c10e67449cb4ef81b10ca97c82, State: Running, Role: LEADER
I20260805 22:42:15.281344 27693 consensus_queue.cc:237] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:15.282106 26875 catalog_manager.cc:5495] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 reported cstate change: term changed from 0 to 1, leader changed from <none> to d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "d252e1c10e67449cb4ef81b10ca97c82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:15.301357 27108 raft_consensus.cc:1261] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Refusing update from remote peer d252e1c10e67449cb4ef81b10ca97c82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:15.301388 27522 raft_consensus.cc:1261] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Refusing update from remote peer d252e1c10e67449cb4ef81b10ca97c82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:15.301627 27689 consensus_queue.cc:1035] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:15.301820 27693 consensus_queue.cc:1035] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:15.303010 27663 mvcc.cc:205] Tried to move back new op lower bound from 7315332035793133568 to 7315332035712856064. Current Snapshot: MvccSnapshot[applied={T|T < 7315332035793133568}]
W20260805 22:42:15.307374 27663 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260805 22:42:16.830750 26875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:16.830873 26875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:16.831864 26875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: Sending DeleteTablet for 3 replicas of tablet 24b3ade6af0b41a491d8b0f31b2bbf74
I20260805 22:42:16.832213 27082 tablet_service.cc:1514] Processing DeleteTablet for tablet 24b3ade6af0b41a491d8b0f31b2bbf74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:16 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:47890
I20260805 22:42:16.832244 27501 tablet_service.cc:1514] Processing DeleteTablet for tablet 24b3ade6af0b41a491d8b0f31b2bbf74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:16 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:53086
I20260805 22:42:16.832224 27288 tablet_service.cc:1514] Processing DeleteTablet for tablet 24b3ade6af0b41a491d8b0f31b2bbf74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:16 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:39180
I20260805 22:42:16.832486 27742 tablet_replica.cc:330] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: stopping tablet replica
I20260805 22:42:16.832549 27742 raft_consensus.cc:2229] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:16.832609 27742 raft_consensus.cc:2258] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:16.832872 27742 ts_tablet_manager.cc:1905] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:16.834852 27742 ts_tablet_manager.cc:1918] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 22:42:16.834919 27742 log.cc:1196] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/24b3ade6af0b41a491d8b0f31b2bbf74
I20260805 22:42:16.834950 27741 tablet_replica.cc:330] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: stopping tablet replica
I20260805 22:42:16.835009 27741 raft_consensus.cc:2229] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:16.835045 27741 raft_consensus.cc:2258] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:16.835171 27740 tablet_replica.cc:330] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: stopping tablet replica
I20260805 22:42:16.835237 27740 raft_consensus.cc:2229] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:16.835283 27740 raft_consensus.cc:2258] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:16.835315 27742 ts_tablet_manager.cc:1939] T 24b3ade6af0b41a491d8b0f31b2bbf74 P d252e1c10e67449cb4ef81b10ca97c82: Deleting consensus metadata
I20260805 22:42:16.835479 27741 ts_tablet_manager.cc:1905] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:16.835641 26859 catalog_manager.cc:4841] TS d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66:40883): tablet 24b3ade6af0b41a491d8b0f31b2bbf74 (table test-table [id=94bec2f51d39426fb258ea7771671468]) successfully deleted
I20260805 22:42:16.837318 27741 ts_tablet_manager.cc:1918] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 22:42:16.837395 27741 log.cc:1196] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/24b3ade6af0b41a491d8b0f31b2bbf74
I20260805 22:42:16.837620 27741 ts_tablet_manager.cc:1939] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 736645fea67f4798ae7f44251332f7d6: Deleting consensus metadata
I20260805 22:42:16.837984 26857 catalog_manager.cc:4841] TS 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203): tablet 24b3ade6af0b41a491d8b0f31b2bbf74 (table test-table [id=94bec2f51d39426fb258ea7771671468]) successfully deleted
I20260805 22:42:16.838093 27740 ts_tablet_manager.cc:1905] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:16.840045 27740 ts_tablet_manager.cc:1918] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 22:42:16.840123 27740 log.cc:1196] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/24b3ade6af0b41a491d8b0f31b2bbf74
I20260805 22:42:16.840385 27740 ts_tablet_manager.cc:1939] T 24b3ade6af0b41a491d8b0f31b2bbf74 P 39f722e7986f4388a43a559e2564c98e: Deleting consensus metadata
I20260805 22:42:16.840744 26856 catalog_manager.cc:4841] TS 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253): tablet 24b3ade6af0b41a491d8b0f31b2bbf74 (table test-table [id=94bec2f51d39426fb258ea7771671468]) successfully deleted
Aug 05 22:42:16 dist-test-slave-g170 krb5kdc[26813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969736, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 22:42:16 dist-test-slave-g170 krb5kdc[26813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969736, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:16.862048 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:16.854182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57328 (local address 127.23.197.126:33671)
0805 22:42:16.854345 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:16.854347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:16.854356 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:16.854383 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:16.854384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:16.854394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:16.854471 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:16.855067 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:16.855597 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:16.856348 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:16.856586 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:16.856686 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:16.859171 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:16.859172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:16.859173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:16.859180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:16.861002 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:16.861348 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:16.861349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:16.861349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:16.861411 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:16.861580 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:16.861580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:16.861581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:16.861691 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:16.861782 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:16.861863 (+ 81us) server_negotiation.cc:299] Negotiation successful
0805 22:42:16.861933 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260805 22:42:16.863976 26875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:16.864137 26875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:16.865747 27288 tablet_service.cc:1467] Processing CreateTablet for tablet 17535eadec11498d939ba794fd90e5be (DEFAULT_TABLE table=test-table [id=d2b7ac9f21084b4eb29c7e4c12c82297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:16.865767 27082 tablet_service.cc:1467] Processing CreateTablet for tablet 17535eadec11498d939ba794fd90e5be (DEFAULT_TABLE table=test-table [id=d2b7ac9f21084b4eb29c7e4c12c82297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:16.865803 27501 tablet_service.cc:1467] Processing CreateTablet for tablet 17535eadec11498d939ba794fd90e5be (DEFAULT_TABLE table=test-table [id=d2b7ac9f21084b4eb29c7e4c12c82297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:16.865892 27288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17535eadec11498d939ba794fd90e5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:16.865916 27501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17535eadec11498d939ba794fd90e5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:16.865916 27082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17535eadec11498d939ba794fd90e5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:16.867304 27757 tablet_bootstrap.cc:492] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: Bootstrap starting.
I20260805 22:42:16.867444 27760 tablet_bootstrap.cc:492] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: Bootstrap starting.
I20260805 22:42:16.867738 27757 tablet_bootstrap.cc:654] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:16.867846 27760 tablet_bootstrap.cc:654] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:16.868147 27758 tablet_bootstrap.cc:492] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: Bootstrap starting.
I20260805 22:42:16.868356 27757 tablet_bootstrap.cc:492] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: No bootstrap required, opened a new log
I20260805 22:42:16.868430 27757 ts_tablet_manager.cc:1397] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:16.868458 27760 tablet_bootstrap.cc:492] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: No bootstrap required, opened a new log
I20260805 22:42:16.868513 27760 ts_tablet_manager.cc:1397] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:16.868520 27758 tablet_bootstrap.cc:654] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:16.868552 27757 raft_consensus.cc:348] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:16.868628 27757 raft_consensus.cc:374] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:16.868655 27757 raft_consensus.cc:729] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d252e1c10e67449cb4ef81b10ca97c82, State: Initialized, Role: FOLLOWER
I20260805 22:42:16.868654 27760 raft_consensus.cc:348] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:16.868737 27760 raft_consensus.cc:374] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:16.868726 27757 consensus_queue.cc:260] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:16.868801 27760 raft_consensus.cc:729] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f722e7986f4388a43a559e2564c98e, State: Initialized, Role: FOLLOWER
I20260805 22:42:16.868856 27760 consensus_queue.cc:260] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:16.869072 27757 ts_tablet_manager.cc:1428] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:16.869096 27760 ts_tablet_manager.cc:1428] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:16.869282 27758 tablet_bootstrap.cc:492] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: No bootstrap required, opened a new log
I20260805 22:42:16.869328 27758 ts_tablet_manager.cc:1397] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:16.869486 27758 raft_consensus.cc:348] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:16.869567 27758 raft_consensus.cc:374] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:16.869603 27758 raft_consensus.cc:729] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736645fea67f4798ae7f44251332f7d6, State: Initialized, Role: FOLLOWER
I20260805 22:42:16.869655 27758 consensus_queue.cc:260] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:16.869829 27758 ts_tablet_manager.cc:1428] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:17.015615 27766 raft_consensus.cc:482] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:17.015739 27766 raft_consensus.cc:504] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.016168 27766 leader_election.cc:288] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66:40883), 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253)
I20260805 22:42:17.020005 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.016389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40883 (local address 127.23.197.65:48687)
0805 22:42:17.016633 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:17.016645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:17.016702 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:17.016915 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:17.016916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:17.016918 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:17.017019 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.017022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.017806 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.017806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.019044 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.019047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.019781 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.019782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.019843 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.019846 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:17.019894 (+ 48us) client_negotiation.cc:240] Negotiation successful
0805 22:42:17.019931 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"spinlock_wait_cycles":768,"thread_start_us":143,"threads_started":1}
I20260805 22:42:17.020072 27773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.016442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:48687 (local address 127.23.197.66:40883)
0805 22:42:17.016736 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.016738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.016746 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:17.016795 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.016796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.016808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.016891 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:17.017093 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.017716 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.019144 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.019692 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.019773 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.019783 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.019923 (+ 140us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.019973 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":91,"threads_started":1}
I20260805 22:42:17.020359 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.016373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:39253 (local address 127.23.197.65:33447)
0805 22:42:17.016609 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:17.016622 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:17.016779 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:17.016997 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:17.016998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:17.017000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:17.017365 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.017368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.018154 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.018155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.019267 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.019269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.020076 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.020077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.020158 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.020163 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:17.020221 (+ 58us) client_negotiation.cc:240] Negotiation successful
0805 22:42:17.020268 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":149,"threads_started":1}
I20260805 22:42:17.020351 27324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17535eadec11498d939ba794fd90e5be" candidate_uuid: "736645fea67f4798ae7f44251332f7d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d252e1c10e67449cb4ef81b10ca97c82" is_pre_election: true
I20260805 22:42:17.020483 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.016471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:33447 (local address 127.23.197.67:39253)
0805 22:42:17.016634 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.016637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.016714 (+ 77us) server_negotiation.cc:407] Connection header received
0805 22:42:17.016854 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.016855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.016908 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.016982 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:17.017454 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.018067 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.019351 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.019976 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.020050 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.020058 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.020327 (+ 269us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.020383 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260805 22:42:17.020493 27324 raft_consensus.cc:2454] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 736645fea67f4798ae7f44251332f7d6 in term 0.
I20260805 22:42:17.020722 27014 leader_election.cc:302] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 736645fea67f4798ae7f44251332f7d6, d252e1c10e67449cb4ef81b10ca97c82; no voters:
I20260805 22:42:17.020709 27526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17535eadec11498d939ba794fd90e5be" candidate_uuid: "736645fea67f4798ae7f44251332f7d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f722e7986f4388a43a559e2564c98e" is_pre_election: true
I20260805 22:42:17.020869 27526 raft_consensus.cc:2454] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 736645fea67f4798ae7f44251332f7d6 in term 0.
I20260805 22:42:17.020892 27766 raft_consensus.cc:2781] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:17.020962 27766 raft_consensus.cc:482] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:17.020988 27766 raft_consensus.cc:3037] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:17.021818 27766 raft_consensus.cc:504] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.022006 27766 leader_election.cc:288] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [CANDIDATE]: Term 1 election: Requested vote from peers d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66:40883), 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253)
I20260805 22:42:17.022177 27324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17535eadec11498d939ba794fd90e5be" candidate_uuid: "736645fea67f4798ae7f44251332f7d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d252e1c10e67449cb4ef81b10ca97c82"
I20260805 22:42:17.022265 27324 raft_consensus.cc:3037] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:17.022205 27526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17535eadec11498d939ba794fd90e5be" candidate_uuid: "736645fea67f4798ae7f44251332f7d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f722e7986f4388a43a559e2564c98e"
I20260805 22:42:17.022320 27526 raft_consensus.cc:3037] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:17.023017 27526 raft_consensus.cc:2454] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 736645fea67f4798ae7f44251332f7d6 in term 1.
I20260805 22:42:17.023042 27324 raft_consensus.cc:2454] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 736645fea67f4798ae7f44251332f7d6 in term 1.
I20260805 22:42:17.023244 27014 leader_election.cc:302] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 736645fea67f4798ae7f44251332f7d6, d252e1c10e67449cb4ef81b10ca97c82; no voters:
I20260805 22:42:17.023347 27766 raft_consensus.cc:2781] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:17.025058 27766 raft_consensus.cc:686] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 1 LEADER]: Becoming Leader. State: Replica: 736645fea67f4798ae7f44251332f7d6, State: Running, Role: LEADER
I20260805 22:42:17.025167 27766 consensus_queue.cc:237] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.026062 26876 catalog_manager.cc:5495] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 736645fea67f4798ae7f44251332f7d6 (127.23.197.65). New cstate: current_term: 1 leader_uuid: "736645fea67f4798ae7f44251332f7d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:17.056082 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.053545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.23.197.65:41203)
0805 22:42:17.053749 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.053752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.053759 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:42:17.053809 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.053809 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.053831 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.053899 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:17.054066 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.054646 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.055363 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.055568 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.055619 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.055796 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.055901 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.055957 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.055997 (+ 40us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.056013 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
I20260805 22:42:17.061298 27773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.058888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:59423 (local address 127.23.197.66:40883)
0805 22:42:17.058967 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.058969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.059155 (+ 186us) server_negotiation.cc:407] Connection header received
0805 22:42:17.059178 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.059179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.059189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.059231 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:17.059370 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.059847 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.060610 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.060844 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.060899 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.061086 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.061112 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.061150 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.061223 (+ 73us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.061243 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 22:42:17.061326 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.058807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40883 (local address 127.23.197.65:59423)
0805 22:42:17.059053 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:17.059064 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:17.059122 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:17.059251 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:17.059251 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:17.059253 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:17.059309 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.059310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.059942 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.059943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.060529 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.060531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.060925 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.060926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.060990 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.061006 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.061168 (+ 162us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.061170 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:42:17.061219 (+ 49us) client_negotiation.cc:240] Negotiation successful
0805 22:42:17.061255 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":148,"threads_started":1}
I20260805 22:42:17.063699 27526 raft_consensus.cc:1261] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Refusing update from remote peer 736645fea67f4798ae7f44251332f7d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:17.063705 27324 raft_consensus.cc:1261] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Refusing update from remote peer 736645fea67f4798ae7f44251332f7d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:17.063989 27775 consensus_queue.cc:1035] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:17.064162 27766 consensus_queue.cc:1035] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:17.072045 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.069771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:50027 (local address 127.23.197.65:41203)
0805 22:42:17.069828 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.069830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.069838 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:17.069859 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.069860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.069870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.069923 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:17.070005 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.070451 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.071343 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.071647 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.071782 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.071886 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.071913 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.071947 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.071977 (+ 30us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.071993 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 22:42:17.072069 27791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.069586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:41203 (local address 127.23.197.66:50027)
0805 22:42:17.069724 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:17.069736 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:17.069784 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:17.069925 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:17.069926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:17.069928 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:17.069968 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.069969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.070543 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.070544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.071248 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.071251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.071780 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.071781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.071827 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.071831 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.071945 (+ 114us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.071946 (+ 1us) client_negotiation.cc:769] Sending connection context
0805 22:42:17.071976 (+ 30us) client_negotiation.cc:240] Negotiation successful
0805 22:42:17.072011 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":87,"threads_started":1}
I20260805 22:42:17.082015 26876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:17.082161 26876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:17.082831 26876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: Sending DeleteTablet for 3 replicas of tablet 17535eadec11498d939ba794fd90e5be
I20260805 22:42:17.083093 27082 tablet_service.cc:1514] Processing DeleteTablet for tablet 17535eadec11498d939ba794fd90e5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:17 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:47890
I20260805 22:42:17.083173 27501 tablet_service.cc:1514] Processing DeleteTablet for tablet 17535eadec11498d939ba794fd90e5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:17 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:53086
I20260805 22:42:17.083231 27741 tablet_replica.cc:330] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: stopping tablet replica
I20260805 22:42:17.083290 27741 raft_consensus.cc:2229] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:17.083364 27288 tablet_service.cc:1514] Processing DeleteTablet for tablet 17535eadec11498d939ba794fd90e5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:17 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:39180
I20260805 22:42:17.083390 27741 raft_consensus.cc:2258] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:17.083453 27742 tablet_replica.cc:330] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: stopping tablet replica
I20260805 22:42:17.083470 27740 tablet_replica.cc:330] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: stopping tablet replica
I20260805 22:42:17.083503 27742 raft_consensus.cc:2229] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:17.083516 27740 raft_consensus.cc:2229] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:17.083540 27742 raft_consensus.cc:2258] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:17.083559 27740 raft_consensus.cc:2258] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:17.083490 26876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:17.083622 26876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:17.083894 27742 ts_tablet_manager.cc:1905] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:17.084123 27741 ts_tablet_manager.cc:1905] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:17.085625 27742 ts_tablet_manager.cc:1918] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:17.085683 27741 ts_tablet_manager.cc:1918] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:17.085709 27742 log.cc:1196] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/17535eadec11498d939ba794fd90e5be
I20260805 22:42:17.085731 27741 log.cc:1196] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/17535eadec11498d939ba794fd90e5be
I20260805 22:42:17.085942 27742 ts_tablet_manager.cc:1939] T 17535eadec11498d939ba794fd90e5be P d252e1c10e67449cb4ef81b10ca97c82: Deleting consensus metadata
I20260805 22:42:17.085942 27741 ts_tablet_manager.cc:1939] T 17535eadec11498d939ba794fd90e5be P 736645fea67f4798ae7f44251332f7d6: Deleting consensus metadata
I20260805 22:42:17.086323 26857 catalog_manager.cc:4841] TS 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203): tablet 17535eadec11498d939ba794fd90e5be (table test-table [id=d2b7ac9f21084b4eb29c7e4c12c82297]) successfully deleted
I20260805 22:42:17.086366 26859 catalog_manager.cc:4841] TS d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66:40883): tablet 17535eadec11498d939ba794fd90e5be (table test-table [id=d2b7ac9f21084b4eb29c7e4c12c82297]) successfully deleted
I20260805 22:42:17.086763 27740 ts_tablet_manager.cc:1905] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:17.087188 27501 tablet_service.cc:1467] Processing CreateTablet for tablet e10595ed386b4cdf992f892332c96a40 (DEFAULT_TABLE table=test-table [id=24eac6bed3f94185a4d8abcc31e31edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:17.087260 27288 tablet_service.cc:1467] Processing CreateTablet for tablet e10595ed386b4cdf992f892332c96a40 (DEFAULT_TABLE table=test-table [id=24eac6bed3f94185a4d8abcc31e31edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:17.087320 27501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e10595ed386b4cdf992f892332c96a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:17.087364 27288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e10595ed386b4cdf992f892332c96a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:17.088346 27740 ts_tablet_manager.cc:1918] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:17.088395 27740 log.cc:1196] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/17535eadec11498d939ba794fd90e5be
I20260805 22:42:17.088613 27740 ts_tablet_manager.cc:1939] T 17535eadec11498d939ba794fd90e5be P 39f722e7986f4388a43a559e2564c98e: Deleting consensus metadata
I20260805 22:42:17.088711 27757 tablet_bootstrap.cc:492] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: Bootstrap starting.
I20260805 22:42:17.088748 27760 tablet_bootstrap.cc:492] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: Bootstrap starting.
I20260805 22:42:17.089053 27757 tablet_bootstrap.cc:654] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:17.089126 27760 tablet_bootstrap.cc:654] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:17.089481 27082 tablet_service.cc:1467] Processing CreateTablet for tablet e10595ed386b4cdf992f892332c96a40 (DEFAULT_TABLE table=test-table [id=24eac6bed3f94185a4d8abcc31e31edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:17.089591 27082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e10595ed386b4cdf992f892332c96a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:17.089643 27757 tablet_bootstrap.cc:492] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: No bootstrap required, opened a new log
I20260805 22:42:17.089711 27757 ts_tablet_manager.cc:1397] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:17.089851 27757 raft_consensus.cc:348] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.089926 27757 raft_consensus.cc:374] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:17.089949 27757 raft_consensus.cc:729] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d252e1c10e67449cb4ef81b10ca97c82, State: Initialized, Role: FOLLOWER
I20260805 22:42:17.089991 27757 consensus_queue.cc:260] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.090091 27757 ts_tablet_manager.cc:1428] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:17.090752 27758 tablet_bootstrap.cc:492] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6: Bootstrap starting.
I20260805 22:42:17.091149 26856 catalog_manager.cc:4841] TS 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253): tablet 17535eadec11498d939ba794fd90e5be (table test-table [id=d2b7ac9f21084b4eb29c7e4c12c82297]) successfully deleted
I20260805 22:42:17.091192 27758 tablet_bootstrap.cc:654] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:17.091492 27760 tablet_bootstrap.cc:492] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: No bootstrap required, opened a new log
I20260805 22:42:17.091538 27760 ts_tablet_manager.cc:1397] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260805 22:42:17.091677 27760 raft_consensus.cc:348] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.091765 27760 raft_consensus.cc:374] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:17.091789 27760 raft_consensus.cc:729] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f722e7986f4388a43a559e2564c98e, State: Initialized, Role: FOLLOWER
I20260805 22:42:17.091835 27760 consensus_queue.cc:260] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.091859 27758 tablet_bootstrap.cc:492] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6: No bootstrap required, opened a new log
I20260805 22:42:17.091907 27758 ts_tablet_manager.cc:1397] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:17.091931 27760 ts_tablet_manager.cc:1428] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:17.092123 27758 raft_consensus.cc:348] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.092207 27758 raft_consensus.cc:374] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:17.092237 27758 raft_consensus.cc:729] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736645fea67f4798ae7f44251332f7d6, State: Initialized, Role: FOLLOWER
I20260805 22:42:17.092281 27758 consensus_queue.cc:260] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.092391 27758 ts_tablet_manager.cc:1428] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:17.261837 27765 raft_consensus.cc:482] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:17.261953 27765 raft_consensus.cc:504] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.262379 27765 leader_election.cc:288] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203), d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66:40883)
I20260805 22:42:17.265494 27773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.262823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:49489 (local address 127.23.197.66:40883)
0805 22:42:17.262899 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.262901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.262913 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:42:17.262945 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.262946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.262958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.263035 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:17.263209 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.263716 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.264690 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.265126 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.265219 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.265229 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.265383 (+ 154us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.265421 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 22:42:17.265512 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.262614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40883 (local address 127.23.197.67:49489)
0805 22:42:17.262778 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:17.262795 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:17.262886 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:17.263040 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:17.263041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:17.263043 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:17.263132 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.263134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.263815 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.263816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.264585 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.264587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.265216 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.265218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.265308 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.265312 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:17.265379 (+ 67us) client_negotiation.cc:240] Negotiation successful
0805 22:42:17.265421 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260805 22:42:17.265615 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.262698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:48941 (local address 127.23.197.65:41203)
0805 22:42:17.262809 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.262812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.262959 (+ 147us) server_negotiation.cc:407] Connection header received
0805 22:42:17.263049 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.263050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.263066 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.263131 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:17.263317 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.263837 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.264859 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.265251 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.265339 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.265347 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.265500 (+ 153us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.265544 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260805 22:42:17.265631 27800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.262614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:41203 (local address 127.23.197.67:48941)
0805 22:42:17.262877 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:17.262892 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:17.262974 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:17.263164 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:17.263165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:17.263167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:17.263244 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.263246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.263966 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.263967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.264725 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.264726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.265338 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.265339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.265419 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.265423 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:17.265493 (+ 70us) client_negotiation.cc:240] Negotiation successful
0805 22:42:17.265534 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":20,"thread_start_us":101,"threads_started":1}
I20260805 22:42:17.266016 27324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e10595ed386b4cdf992f892332c96a40" candidate_uuid: "39f722e7986f4388a43a559e2564c98e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d252e1c10e67449cb4ef81b10ca97c82" is_pre_election: true
I20260805 22:42:17.266032 27108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e10595ed386b4cdf992f892332c96a40" candidate_uuid: "39f722e7986f4388a43a559e2564c98e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736645fea67f4798ae7f44251332f7d6" is_pre_election: true
I20260805 22:42:17.266124 27108 raft_consensus.cc:2454] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f722e7986f4388a43a559e2564c98e in term 0.
I20260805 22:42:17.266124 27324 raft_consensus.cc:2454] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f722e7986f4388a43a559e2564c98e in term 0.
I20260805 22:42:17.266312 27435 leader_election.cc:302] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f722e7986f4388a43a559e2564c98e, d252e1c10e67449cb4ef81b10ca97c82; no voters:
I20260805 22:42:17.266462 27765 raft_consensus.cc:2781] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:17.266517 27765 raft_consensus.cc:482] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:17.266532 27765 raft_consensus.cc:3037] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:17.267299 27765 raft_consensus.cc:504] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.267447 27765 leader_election.cc:288] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [CANDIDATE]: Term 1 election: Requested vote from peers 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203), d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66:40883)
I20260805 22:42:17.267604 27108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e10595ed386b4cdf992f892332c96a40" candidate_uuid: "39f722e7986f4388a43a559e2564c98e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736645fea67f4798ae7f44251332f7d6"
I20260805 22:42:17.267629 27324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e10595ed386b4cdf992f892332c96a40" candidate_uuid: "39f722e7986f4388a43a559e2564c98e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d252e1c10e67449cb4ef81b10ca97c82"
I20260805 22:42:17.267694 27108 raft_consensus.cc:3037] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:17.267742 27324 raft_consensus.cc:3037] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:17.268524 27108 raft_consensus.cc:2454] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f722e7986f4388a43a559e2564c98e in term 1.
I20260805 22:42:17.268524 27324 raft_consensus.cc:2454] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f722e7986f4388a43a559e2564c98e in term 1.
I20260805 22:42:17.268728 27433 leader_election.cc:302] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f722e7986f4388a43a559e2564c98e, 736645fea67f4798ae7f44251332f7d6; no voters:
I20260805 22:42:17.268887 27765 raft_consensus.cc:2781] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:17.269033 27765 raft_consensus.cc:686] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 1 LEADER]: Becoming Leader. State: Replica: 39f722e7986f4388a43a559e2564c98e, State: Running, Role: LEADER
I20260805 22:42:17.269112 27765 consensus_queue.cc:237] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } }
I20260805 22:42:17.269882 26876 catalog_manager.cc:5495] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e reported cstate change: term changed from 0 to 1, leader changed from <none> to 39f722e7986f4388a43a559e2564c98e (127.23.197.67). New cstate: current_term: 1 leader_uuid: "39f722e7986f4388a43a559e2564c98e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f722e7986f4388a43a559e2564c98e" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 39253 } health_report { overall_health: HEALTHY } } }
I20260805 22:42:17.279783 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.277299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36802 (local address 127.23.197.67:39253)
0805 22:42:17.277340 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.277342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.277350 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:17.277393 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.277393 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.277416 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.277511 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:17.277630 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.278112 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.278911 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.279137 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.279231 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.279405 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.279529 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.279622 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.279663 (+ 41us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.279686 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:42:17.285480 27773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.283121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:53405 (local address 127.23.197.66:40883)
0805 22:42:17.283186 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.283188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.283241 (+ 53us) server_negotiation.cc:407] Connection header received
0805 22:42:17.283312 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.283313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.283322 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.283361 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:17.283490 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.284032 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.284860 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.285047 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.285104 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.285288 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.285314 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.285353 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.285406 (+ 53us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.285424 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 22:42:17.285514 27807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.282931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40883 (local address 127.23.197.67:53405)
0805 22:42:17.283176 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:17.283196 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:17.283253 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:17.283362 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:17.283362 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:17.283364 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:17.283442 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.283443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.284113 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.284114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.284698 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.284700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.285135 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.285137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.285190 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.285209 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.285368 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.285370 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:42:17.285405 (+ 35us) client_negotiation.cc:240] Negotiation successful
0805 22:42:17.285446 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":106,"threads_started":1}
I20260805 22:42:17.287744 27324 raft_consensus.cc:1261] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Refusing update from remote peer 39f722e7986f4388a43a559e2564c98e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:17.287755 27108 raft_consensus.cc:1261] T e10595ed386b4cdf992f892332c96a40 P 736645fea67f4798ae7f44251332f7d6 [term 1 FOLLOWER]: Refusing update from remote peer 39f722e7986f4388a43a559e2564c98e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:17.288089 27801 consensus_queue.cc:1035] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d252e1c10e67449cb4ef81b10ca97c82" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:17.288236 27765 consensus_queue.cc:1035] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [LEADER]: Connected to new peer: Peer: permanent_uuid: "736645fea67f4798ae7f44251332f7d6" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 41203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:17.295619 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.293304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:59071 (local address 127.23.197.67:39253)
0805 22:42:17.293348 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:17.293350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:17.293430 (+ 80us) server_negotiation.cc:407] Connection header received
0805 22:42:17.293480 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:17.293481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:17.293494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:17.293554 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:17.293684 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.294165 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.294875 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.295173 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.295241 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.295393 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.295429 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.295481 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:17.295543 (+ 62us) server_negotiation.cc:299] Negotiation successful
0805 22:42:17.295569 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:42:17.295619 27791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:17.293253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:39253 (local address 127.23.197.66:59071)
0805 22:42:17.293378 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:17.293389 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:17.293439 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:17.293566 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:17.293568 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:17.293569 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:17.293615 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.293616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.294230 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.294231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.294749 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:17.294750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:17.295247 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:17.295248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:17.295311 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:17.295319 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:17.295495 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:17.295497 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:42:17.295530 (+ 33us) client_negotiation.cc:240] Negotiation successful
0805 22:42:17.295567 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
W20260805 22:42:17.296316 27717 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260805 22:42:18.822718 26875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:18.822868 26875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:18.824002 26875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 343a319865b249b2a2a9e7cd009f2ea4: Sending DeleteTablet for 3 replicas of tablet e10595ed386b4cdf992f892332c96a40
I20260805 22:42:18.824337 27501 tablet_service.cc:1514] Processing DeleteTablet for tablet e10595ed386b4cdf992f892332c96a40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:18 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:53086
I20260805 22:42:18.824362 27290 tablet_service.cc:1514] Processing DeleteTablet for tablet e10595ed386b4cdf992f892332c96a40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:18 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:39180
I20260805 22:42:18.824386 27082 tablet_service.cc:1514] Processing DeleteTablet for tablet e10595ed386b4cdf992f892332c96a40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:18 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:47890
I20260805 22:42:18.824587 27851 tablet_replica.cc:330] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: stopping tablet replica
I20260805 22:42:18.824656 27851 raft_consensus.cc:2229] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:18.824715 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 26984
I20260805 22:42:18.824795 27851 raft_consensus.cc:2258] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:18.837145 27849 tablet_replica.cc:330] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: stopping tablet replica
I20260805 22:42:18.837272 27849 raft_consensus.cc:2229] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:18.837322 27851 ts_tablet_manager.cc:1905] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:18.837438 27849 raft_consensus.cc:2258] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:18.837841 27849 ts_tablet_manager.cc:1905] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:18.839483 27849 ts_tablet_manager.cc:1918] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 22:42:18.839489 27851 ts_tablet_manager.cc:1918] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 22:42:18.839589 27851 log.cc:1196] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/e10595ed386b4cdf992f892332c96a40
I20260805 22:42:18.839596 27849 log.cc:1196] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/e10595ed386b4cdf992f892332c96a40
I20260805 22:42:18.839830 27851 ts_tablet_manager.cc:1939] T e10595ed386b4cdf992f892332c96a40 P d252e1c10e67449cb4ef81b10ca97c82: Deleting consensus metadata
I20260805 22:42:18.840071 27849 ts_tablet_manager.cc:1939] T e10595ed386b4cdf992f892332c96a40 P 39f722e7986f4388a43a559e2564c98e: Deleting consensus metadata
I20260805 22:42:18.840438 26859 catalog_manager.cc:4841] TS d252e1c10e67449cb4ef81b10ca97c82 (127.23.197.66:40883): tablet e10595ed386b4cdf992f892332c96a40 (table test-table [id=24eac6bed3f94185a4d8abcc31e31edd]) successfully deleted
I20260805 22:42:18.840504 26856 catalog_manager.cc:4841] TS 39f722e7986f4388a43a559e2564c98e (127.23.197.67:39253): tablet e10595ed386b4cdf992f892332c96a40 (table test-table [id=24eac6bed3f94185a4d8abcc31e31edd]) successfully deleted
W20260805 22:42:18.842223 26857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:41203 (error 108)
I20260805 22:42:18.842473 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 27189
I20260805 22:42:18.846630 27853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:18.842621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:41203 (local address 127.0.0.1:60302)
0805 22:42:18.846500 (+ 3879us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:18.846573 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3845,"thread_start_us":68,"threads_started":1}
W20260805 22:42:18.846756 26857 catalog_manager.cc:4584] TS 736645fea67f4798ae7f44251332f7d6 (127.23.197.65:41203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e10595ed386b4cdf992f892332c96a40: Network error: Client connection negotiation failed: client connection to 127.23.197.65:41203: connect: Connection refused (error 111)
I20260805 22:42:18.862371 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 27406
I20260805 22:42:18.877215 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 26831
2026-08-05T22:42:18Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6280 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:18 dist-test-slave-g170 krb5kdc[27857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:18 dist-test-slave-g170 krb5kdc[27857](info): set up 2 sockets
Aug 05 22:42:18 dist-test-slave-g170 krb5kdc[27857](info): commencing operation
krb5kdc: starting...
W20260805 22:42:20.175819 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:20 dist-test-slave-g170 krb5kdc[27857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:20.308139 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:44333
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:44333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:20.392874 27873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:20.393072 27873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:20.393095 27873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:20.394325 27873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:20.394371 27873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:20.394385 27873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:20.394398 27873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:20.394438 27873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:20.395928 27873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:44333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:44333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:20.396180 27873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:20.396428 27873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:20.401932 27888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:20.401971 27880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:20.401932 27879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:20.401974 27873 server_base.cc:1034] running on GCE node
I20260805 22:42:20.402575 27873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:20.402948 27873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:20.404120 27873 hybrid_clock.cc:648] HybridClock initialized: now 1785969740404110 us; error 51 us; skew 500 ppm
Aug 05 22:42:20 dist-test-slave-g170 krb5kdc[27857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:20.411808 27873 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:20.412449 27873 webserver.cc:460] Webserver started at http://127.23.197.126:44103/ using document root <none> and password file <none>
I20260805 22:42:20.412798 27873 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:20.412911 27873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:20.413081 27873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:20.414453 27873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "778b5e5ab1f3421fac2952be32c494ff"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "7b3cb6b6ee6ae60ff77ba18fb611769e"
server_key_iv: "3ba1f440331e4f33893ec840cb8f7332"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.414868 27873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "778b5e5ab1f3421fac2952be32c494ff"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "7b3cb6b6ee6ae60ff77ba18fb611769e"
server_key_iv: "3ba1f440331e4f33893ec840cb8f7332"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.416366 27873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260805 22:42:20.417340 27900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:20.417613 27873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:42:20.417725 27873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "778b5e5ab1f3421fac2952be32c494ff"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "7b3cb6b6ee6ae60ff77ba18fb611769e"
server_key_iv: "3ba1f440331e4f33893ec840cb8f7332"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.417801 27873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:20.432749 27873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:20.435336 27873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:20.435467 27873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:20.439879 27873 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:44333
I20260805 22:42:20.439954 27974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:44333 every 8 connection(s)
I20260805 22:42:20.440358 27873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:20.441411 27975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:20.443797 27975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff: Bootstrap starting.
I20260805 22:42:20.444401 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 27873
I20260805 22:42:20.444435 27975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:20.444481 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:20.444684 24341 external_mini_cluster.cc:1428] Setting key 51169c9cc440cc25dd518ba59c3b5cb4
I20260805 22:42:20.444773 27975 log.cc:824] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:20.445633 27975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff: No bootstrap required, opened a new log
I20260805 22:42:20.447717 27975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778b5e5ab1f3421fac2952be32c494ff" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44333 } }
I20260805 22:42:20.447881 27975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:20.447932 27975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 778b5e5ab1f3421fac2952be32c494ff, State: Initialized, Role: FOLLOWER
I20260805 22:42:20.448062 27975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778b5e5ab1f3421fac2952be32c494ff" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44333 } }
I20260805 22:42:20.448139 27975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:20.448163 27975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:20.448233 27975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:20.448840 27975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778b5e5ab1f3421fac2952be32c494ff" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44333 } }
I20260805 22:42:20.448968 27975 leader_election.cc:302] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 778b5e5ab1f3421fac2952be32c494ff; no voters:
I20260805 22:42:20.449153 27975 leader_election.cc:288] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:20.449193 27985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:20.449357 27985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [term 1 LEADER]: Becoming Leader. State: Replica: 778b5e5ab1f3421fac2952be32c494ff, State: Running, Role: LEADER
I20260805 22:42:20.449419 27975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:20.449457 27985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778b5e5ab1f3421fac2952be32c494ff" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44333 } }
I20260805 22:42:20.449774 27986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "778b5e5ab1f3421fac2952be32c494ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778b5e5ab1f3421fac2952be32c494ff" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44333 } } }
I20260805 22:42:20.449858 27986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:20.450023 27989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 778b5e5ab1f3421fac2952be32c494ff. Latest consensus state: current_term: 1 leader_uuid: "778b5e5ab1f3421fac2952be32c494ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778b5e5ab1f3421fac2952be32c494ff" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44333 } } }
I20260805 22:42:20.450088 27989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:20.450426 27991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:20.450919 27991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 05 22:42:20 dist-test-slave-g170 krb5kdc[27857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:20.452562 27991 catalog_manager.cc:1347] Generated new cluster ID: 666211dc1dc74c0db8623fa17433d4fa
I20260805 22:42:20.452616 27991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:20.456151 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:20.445285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.23.197.126:44333)
0805 22:42:20.445500 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:20.445511 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:20.445544 (+ 33us) server_negotiation.cc:407] Connection header received
0805 22:42:20.446014 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:20.446022 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:20.446073 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:20.446220 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:20.446882 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.447745 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.448679 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.449293 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.449410 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:20.452681 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:20.452684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:20.452685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:20.452697 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:20.454556 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:20.454950 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:20.454953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:20.454953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:20.455047 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:20.455262 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:20.455263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:20.455264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:20.455501 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:20.455689 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:20.455789 (+ 100us) server_negotiation.cc:299] Negotiation successful
0805 22:42:20.455928 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260805 22:42:20.477592 27991 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:20.478132 27991 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:20.484655 27991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 778b5e5ab1f3421fac2952be32c494ff: Generated new TSK 0
I20260805 22:42:20.484876 27991 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:20.529055 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44333
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:20.637432 28021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:20.637770 28021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:20.637832 28021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:20.639178 28021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:20.639256 28021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:20.639305 28021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:20.640959 28021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:20.641250 28021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:20.641513 28021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 22:42:20.647135 28021 server_base.cc:1034] running on GCE node
W20260805 22:42:20.647089 28030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:20.647109 28033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:20.647107 28029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:20.647637 28021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:20.647853 28021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:20.649005 28021 hybrid_clock.cc:648] HybridClock initialized: now 1785969740648993 us; error 31 us; skew 500 ppm
Aug 05 22:42:20 dist-test-slave-g170 krb5kdc[27857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:20.651899 28021 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:20.652379 28021 webserver.cc:460] Webserver started at http://127.23.197.65:38443/ using document root <none> and password file <none>
I20260805 22:42:20.652611 28021 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:20.652657 28021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:20.652851 28021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:20.653940 28021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "b9a0da6d75bd41128e57541a5de09dc4"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "94213a4cb40a01c0da862727d6afcd89"
server_key_iv: "de30bfb49c0571bccc31d47209f29184"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.654316 28021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "b9a0da6d75bd41128e57541a5de09dc4"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "94213a4cb40a01c0da862727d6afcd89"
server_key_iv: "de30bfb49c0571bccc31d47209f29184"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.655756 28021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:20.656500 28045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:20.656697 28021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:20.656836 28021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "b9a0da6d75bd41128e57541a5de09dc4"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "94213a4cb40a01c0da862727d6afcd89"
server_key_iv: "de30bfb49c0571bccc31d47209f29184"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.656903 28021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:20.670531 28021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:20.672585 28021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:20.672703 28021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:20.672976 28021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:20.673323 28021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:20.673358 28021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:20.673381 28021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:20.673431 28021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:20.679327 28021 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:43427
I20260805 22:42:20.679399 28217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:43427 every 8 connection(s)
I20260805 22:42:20.679740 28021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:20.686302 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 28021
I20260805 22:42:20.686424 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:20.686618 24341 external_mini_cluster.cc:1428] Setting key be0b10669e202beaf0ac0d0dfc85e7a3
Aug 05 22:42:20 dist-test-slave-g170 krb5kdc[27857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:20.690608 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:20.680648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:59347 (local address 127.23.197.126:44333)
0805 22:42:20.680729 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:20.680732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:20.680972 (+ 240us) server_negotiation.cc:407] Connection header received
0805 22:42:20.681567 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:20.681569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:20.681583 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:20.681717 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:20.682813 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.683374 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.684503 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.684751 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.684973 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:20.687639 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:20.687641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:20.687641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:20.687649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:20.689153 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:20.689571 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:20.689573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:20.689573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:20.689640 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:20.689921 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:20.689922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:20.689922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:20.690049 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:20.690124 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:20.690426 (+ 302us) server_negotiation.cc:299] Negotiation successful
0805 22:42:20.690516 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 22:42:20.690805 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:20.680521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:44333 (local address 127.23.197.65:59347)
0805 22:42:20.680810 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:20.680872 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:20.681446 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:20.681798 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:20.681803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:20.682476 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:20.682688 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:20.682693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.683494 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.683495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:20.684391 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:20.684393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.684983 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.684985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:20.685135 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:20.685692 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:20.685702 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:20.687525 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:20.689244 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:20.689252 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:20.689255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:20.689487 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:20.689738 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:20.689739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:20.689739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:20.689853 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:20.690126 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:20.690134 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:20.690286 (+ 152us) client_negotiation.cc:769] Sending connection context
0805 22:42:20.690393 (+ 107us) client_negotiation.cc:240] Negotiation successful
0805 22:42:20.690559 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":72,"threads_started":1}
I20260805 22:42:20.691309 28219 heartbeater.cc:347] Connected to a master server at 127.23.197.126:44333
I20260805 22:42:20.691434 28219 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:20.691653 28219 heartbeater.cc:511] Master 127.23.197.126:44333 requested a full tablet report, sending...
I20260805 22:42:20.692219 27922 ts_manager.cc:194] Registered new tserver with Master: b9a0da6d75bd41128e57541a5de09dc4 (127.23.197.65:43427)
I20260805 22:42:20.692991 27922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:59347
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:20.716300 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44333
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:20.804116 28225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:20.804447 28225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:20.804486 28225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:20.805909 28225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:20.805958 28225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:20.805987 28225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:20.807500 28225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:20.807758 28225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:20.808004 28225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:20.813259 28235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:20.813268 28238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:20.813268 28233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:20.813519 28225 server_base.cc:1034] running on GCE node
I20260805 22:42:20.813675 28225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:20.813887 28225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:20.815034 28225 hybrid_clock.cc:648] HybridClock initialized: now 1785969740815018 us; error 33 us; skew 500 ppm
Aug 05 22:42:20 dist-test-slave-g170 krb5kdc[27857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:20.817819 28225 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:20.818297 28225 webserver.cc:460] Webserver started at http://127.23.197.66:37923/ using document root <none> and password file <none>
I20260805 22:42:20.818562 28225 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:20.818609 28225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:20.818754 28225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:20.819873 28225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "f05d0961977c4abcbd122faa04aafc99"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "1076c35763b1214c690d263e09e6d5aa"
server_key_iv: "c2791e7fd1b758a48f34ec9d7f4d2408"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.820318 28225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "f05d0961977c4abcbd122faa04aafc99"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "1076c35763b1214c690d263e09e6d5aa"
server_key_iv: "c2791e7fd1b758a48f34ec9d7f4d2408"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.821925 28225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260805 22:42:20.822814 28250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:20.823040 28225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:20.823124 28225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "f05d0961977c4abcbd122faa04aafc99"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "1076c35763b1214c690d263e09e6d5aa"
server_key_iv: "c2791e7fd1b758a48f34ec9d7f4d2408"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.823215 28225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:20.839356 28225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:20.841917 28225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:20.842072 28225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:20.842360 28225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:20.842702 28225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:20.842746 28225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:20.842770 28225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:20.842783 28225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:20.849599 28225 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:42163
I20260805 22:42:20.849653 28433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:42163 every 8 connection(s)
I20260805 22:42:20.850003 28225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:20.852612 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 28225
I20260805 22:42:20.852739 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:20.852965 24341 external_mini_cluster.cc:1428] Setting key 3a5ce97d499b0b6643270c1423ccff80
Aug 05 22:42:20 dist-test-slave-g170 krb5kdc[27857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:20.861320 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:20.850799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:56645 (local address 127.23.197.126:44333)
0805 22:42:20.850885 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:20.850887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:20.851123 (+ 236us) server_negotiation.cc:407] Connection header received
0805 22:42:20.851768 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:20.851769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:20.851784 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:20.851836 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:20.853117 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.853732 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.855322 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.855516 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.855578 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:20.858057 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:20.858058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:20.858059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:20.858067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:20.859758 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:20.860233 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:20.860235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:20.860235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:20.860300 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:20.860570 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:20.860571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:20.860571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:20.860733 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:20.860831 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:20.861175 (+ 344us) server_negotiation.cc:299] Negotiation successful
0805 22:42:20.861250 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 22:42:20.861624 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:20.850817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:44333 (local address 127.23.197.66:56645)
0805 22:42:20.850986 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:20.851027 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:20.851640 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:20.851914 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:20.851917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:20.852665 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:20.853005 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:20.853013 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.853817 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.853818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:20.855216 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:20.855220 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:20.855608 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:20.855608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:20.855781 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:20.856400 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:20.856410 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:20.857949 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:20.859865 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:20.859874 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:20.859876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:20.860115 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:20.860381 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:20.860382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:20.860382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:20.860492 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:20.860829 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:20.860837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:20.861011 (+ 174us) client_negotiation.cc:769] Sending connection context
0805 22:42:20.861140 (+ 129us) client_negotiation.cc:240] Negotiation successful
0805 22:42:20.861327 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260805 22:42:20.862326 28434 heartbeater.cc:347] Connected to a master server at 127.23.197.126:44333
I20260805 22:42:20.862437 28434 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:20.862658 28434 heartbeater.cc:511] Master 127.23.197.126:44333 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
I20260805 22:42:20.863116 27922 ts_manager.cc:194] Registered new tserver with Master: f05d0961977c4abcbd122faa04aafc99 (127.23.197.66:42163)
I20260805 22:42:20.863549 27922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:56645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:20.884184 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44333
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:20.970666 28439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:20.970927 28439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:20.970970 28439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:20.972196 28439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:20.972261 28439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:20.972307 28439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:20.973884 28439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:20.974176 28439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:20.974421 28439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:20.979709 28446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:20.979712 28449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:20.979709 28447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:20.980159 28439 server_base.cc:1034] running on GCE node
I20260805 22:42:20.980455 28439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:20.980849 28439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:20.982040 28439 hybrid_clock.cc:648] HybridClock initialized: now 1785969740982021 us; error 33 us; skew 500 ppm
Aug 05 22:42:20 dist-test-slave-g170 krb5kdc[27857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:20.985013 28439 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:20.985517 28439 webserver.cc:460] Webserver started at http://127.23.197.67:38745/ using document root <none> and password file <none>
I20260805 22:42:20.985796 28439 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:20.985849 28439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:20.985999 28439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:20.987174 28439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "0c6853678e734aa59e9391e36ac6635f"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "a198e1304f3f3c1b56a92ca14c5ac54f"
server_key_iv: "faf322cec303929234d5ad1fd02d843f"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.987582 28439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "0c6853678e734aa59e9391e36ac6635f"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "a198e1304f3f3c1b56a92ca14c5ac54f"
server_key_iv: "faf322cec303929234d5ad1fd02d843f"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.989187 28439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:20.990060 28462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:20.990255 28439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:20.990360 28439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "0c6853678e734aa59e9391e36ac6635f"
format_stamp: "Formatted at 2026-08-05 22:42:20 on dist-test-slave-g170"
server_key: "a198e1304f3f3c1b56a92ca14c5ac54f"
server_key_iv: "faf322cec303929234d5ad1fd02d843f"
server_key_version: "encryptionkey@0"
I20260805 22:42:20.990418 28439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:21.007222 28439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:21.009917 28439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:21.010043 28439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:21.010309 28439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:21.010605 28439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:21.010636 28439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:21.010659 28439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:21.010671 28439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:21.017132 28439 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:42165
I20260805 22:42:21.017199 28633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:42165 every 8 connection(s)
I20260805 22:42:21.017580 28439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:21.020524 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 28439
I20260805 22:42:21.020644 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:21.020862 24341 external_mini_cluster.cc:1428] Setting key 8bb2cb1a651516317c83068b6670ef65
Aug 05 22:42:21 dist-test-slave-g170 krb5kdc[27857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:21.028575 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:21.018487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:58763 (local address 127.23.197.126:44333)
0805 22:42:21.018582 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:21.018584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:21.018844 (+ 260us) server_negotiation.cc:407] Connection header received
0805 22:42:21.019297 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:21.019298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:21.019310 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:21.019385 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:21.020387 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:21.021228 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:21.022482 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:21.022678 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:21.022774 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:21.025366 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:21.025369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:21.025369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:21.025376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:21.026935 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:21.027456 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:21.027458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:21.027458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:21.027532 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:21.027866 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:21.027867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:21.027867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:21.028000 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:21.028076 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:21.028381 (+ 305us) server_negotiation.cc:299] Negotiation successful
0805 22:42:21.028475 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 22:42:21.028847 28637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:21.018469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:44333 (local address 127.23.197.67:58763)
0805 22:42:21.018666 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:21.018717 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:21.019208 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:21.019440 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:21.019444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:21.020074 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:21.020277 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:21.020282 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:21.021313 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:21.021314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:21.022399 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:21.022402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:21.022812 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:21.022814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:21.023087 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:21.023603 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:21.023612 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:21.025268 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:21.027022 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:21.027029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:21.027032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:21.027274 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:21.027611 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:21.027612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:21.027612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:21.027776 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:21.028080 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:21.028089 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:21.028238 (+ 149us) client_negotiation.cc:769] Sending connection context
0805 22:42:21.028357 (+ 119us) client_negotiation.cc:240] Negotiation successful
0805 22:42:21.028539 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260805 22:42:21.029404 28634 heartbeater.cc:347] Connected to a master server at 127.23.197.126:44333
I20260805 22:42:21.029520 28634 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:21.029721 28634 heartbeater.cc:511] Master 127.23.197.126:44333 requested a full tablet report, sending...
I20260805 22:42:21.030118 27922 ts_manager.cc:194] Registered new tserver with Master: 0c6853678e734aa59e9391e36ac6635f (127.23.197.67:42165)
I20260805 22:42:21.030474 27922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:58763
I20260805 22:42:21.034669 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260805 22:42:21.047364 27980 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:21.046353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50158 (local address 127.23.197.126:44333)
0805 22:42:21.046452 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:21.046454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:21.046577 (+ 123us) server_negotiation.cc:407] Connection header received
0805 22:42:21.046608 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:21.046609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:21.046620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:21.046666 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:21.047276 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50158: BlockingRecv error: recv got EOF from 127.0.0.1:50158 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 22:42:21.048352 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 28021
I20260805 22:42:21.060874 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 28225
I20260805 22:42:21.070127 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 28439
I20260805 22:42:21.079861 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 27873
2026-08-05T22:42:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2197 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:21 dist-test-slave-g170 krb5kdc[28648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:21 dist-test-slave-g170 krb5kdc[28648](info): set up 2 sockets
Aug 05 22:42:21 dist-test-slave-g170 krb5kdc[28648](info): commencing operation
krb5kdc: starting...
W20260805 22:42:22.338832 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:22 dist-test-slave-g170 krb5kdc[28648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:22.468375 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38917
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:33445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:38917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:22.554275 28671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:22.554553 28671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:22.554595 28671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:22.555855 28671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:22.555933 28671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:22.555958 28671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:22.555979 28671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:22.556006 28671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:22.557775 28671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:33445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:38917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:22.558080 28671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:22.558355 28671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:22.563668 28681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:22.563681 28676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:22.563772 28671 server_base.cc:1034] running on GCE node
W20260805 22:42:22.563704 28678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:22.564294 28671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:22.564625 28671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:22.565831 28671 hybrid_clock.cc:648] HybridClock initialized: now 1785969742565822 us; error 45 us; skew 500 ppm
Aug 05 22:42:22 dist-test-slave-g170 krb5kdc[28648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969742, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:22.568521 28671 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:22.569080 28671 webserver.cc:460] Webserver started at http://127.23.197.126:33395/ using document root <none> and password file <none>
I20260805 22:42:22.569320 28671 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:22.569392 28671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:22.569548 28671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:22.570668 28671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "e46909775ecc40b892d52fac2da38803"
format_stamp: "Formatted at 2026-08-05 22:42:22 on dist-test-slave-g170"
server_key: "241f08002dd03a3514e7c7e84e5ce700"
server_key_iv: "eb9a1d59729bb68ddcd0ac7e727d7b5d"
server_key_version: "encryptionkey@0"
I20260805 22:42:22.571071 28671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "e46909775ecc40b892d52fac2da38803"
format_stamp: "Formatted at 2026-08-05 22:42:22 on dist-test-slave-g170"
server_key: "241f08002dd03a3514e7c7e84e5ce700"
server_key_iv: "eb9a1d59729bb68ddcd0ac7e727d7b5d"
server_key_version: "encryptionkey@0"
I20260805 22:42:22.572605 28671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:22.573519 28690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:22.573794 28671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:22.573920 28671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "e46909775ecc40b892d52fac2da38803"
format_stamp: "Formatted at 2026-08-05 22:42:22 on dist-test-slave-g170"
server_key: "241f08002dd03a3514e7c7e84e5ce700"
server_key_iv: "eb9a1d59729bb68ddcd0ac7e727d7b5d"
server_key_version: "encryptionkey@0"
I20260805 22:42:22.573999 28671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:22.589840 28671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:22.592229 28671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:22.592443 28671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:22.596926 28671 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:38917
I20260805 22:42:22.596964 28782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:38917 every 8 connection(s)
I20260805 22:42:22.597549 28671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:22.598369 28785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:22.600996 28785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803: Bootstrap starting.
I20260805 22:42:22.601656 28785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:22.601989 28785 log.cc:824] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:22.602805 28785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803: No bootstrap required, opened a new log
I20260805 22:42:22.604338 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 28671
I20260805 22:42:22.604476 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:22.604538 28785 raft_consensus.cc:348] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46909775ecc40b892d52fac2da38803" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38917 } }
I20260805 22:42:22.604709 28785 raft_consensus.cc:374] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:22.604728 24341 external_mini_cluster.cc:1428] Setting key 0e35222a07fa101f3ecdedc26476cd2a
I20260805 22:42:22.604743 28785 raft_consensus.cc:729] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e46909775ecc40b892d52fac2da38803, State: Initialized, Role: FOLLOWER
I20260805 22:42:22.604943 28785 consensus_queue.cc:260] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46909775ecc40b892d52fac2da38803" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38917 } }
I20260805 22:42:22.605036 28785 raft_consensus.cc:388] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:22.605060 28785 raft_consensus.cc:482] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:22.605127 28785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:22.605852 28785 raft_consensus.cc:504] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46909775ecc40b892d52fac2da38803" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38917 } }
I20260805 22:42:22.605975 28785 leader_election.cc:302] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e46909775ecc40b892d52fac2da38803; no voters:
I20260805 22:42:22.606168 28785 leader_election.cc:288] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:22.606212 28791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:22.606362 28791 raft_consensus.cc:686] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [term 1 LEADER]: Becoming Leader. State: Replica: e46909775ecc40b892d52fac2da38803, State: Running, Role: LEADER
I20260805 22:42:22.606458 28785 sys_catalog.cc:564] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:22.606487 28791 consensus_queue.cc:237] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46909775ecc40b892d52fac2da38803" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38917 } }
I20260805 22:42:22.606926 28791 sys_catalog.cc:455] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e46909775ecc40b892d52fac2da38803. Latest consensus state: current_term: 1 leader_uuid: "e46909775ecc40b892d52fac2da38803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46909775ecc40b892d52fac2da38803" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38917 } } }
I20260805 22:42:22.607033 28791 sys_catalog.cc:458] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:22.606940 28793 sys_catalog.cc:455] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e46909775ecc40b892d52fac2da38803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46909775ecc40b892d52fac2da38803" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38917 } } }
I20260805 22:42:22.607340 28793 sys_catalog.cc:458] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803 [sys.catalog]: This master's current role is: LEADER
W20260805 22:42:22.608641 28810 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:42:22.608701 28810 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:42:22.608779 28801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:22.608965 28801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:22.610497 28801 catalog_manager.cc:1347] Generated new cluster ID: b8f3623a3700411b86bbc786fe970cb8
I20260805 22:42:22.610558 28801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:42:22 dist-test-slave-g170 krb5kdc[28648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:22.617686 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:22.605358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.23.197.126:38917)
0805 22:42:22.605937 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:22.605948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:22.605982 (+ 34us) server_negotiation.cc:407] Connection header received
0805 22:42:22.606141 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:22.606147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:22.606222 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:22.606385 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:22.607583 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:22.608856 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:22.609930 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:22.610490 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:22.610618 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:22.613094 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:22.613099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:22.613099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:22.613120 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:22.615804 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:22.616322 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:22.616325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:22.616326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:22.616435 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:22.616669 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:22.616670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:22.616671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:22.617008 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:22.617216 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:22.617330 (+ 114us) server_negotiation.cc:299] Negotiation successful
0805 22:42:22.617484 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":518,"thread_start_us":484,"threads_started":1}
I20260805 22:42:22.618721 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:42:22.634749 28801 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:22.635622 28801 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:22.638705 28801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e46909775ecc40b892d52fac2da38803: Generated new TSK 0
I20260805 22:42:22.638902 28801 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:42:22.640714 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:22.636531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.23.197.126:38917)
0805 22:42:22.636573 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:22.636576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:22.636587 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:22.636656 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:22.636657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:22.636669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:22.636724 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:22.637207 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:22.637970 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:22.639028 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:22.639377 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:22.639454 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:22.640233 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:22.640234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:22.640234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:22.640463 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:22.640483 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:22.640491 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:22.640556 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:22.640606 (+ 50us) server_negotiation.cc:299] Negotiation successful
0805 22:42:22.640635 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260805 22:42:22.643216 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 28671
2026-08-05T22:42:22Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1562 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:22 dist-test-slave-g170 krb5kdc[28835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:22 dist-test-slave-g170 krb5kdc[28835](info): set up 2 sockets
Aug 05 22:42:22 dist-test-slave-g170 krb5kdc[28835](info): commencing operation
krb5kdc: starting...
W20260805 22:42:23.911402 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:24.060407 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33333
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:33333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:24.148633 28855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:24.148940 28855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:24.148988 28855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:24.150223 28855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:24.150295 28855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:24.150319 28855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:24.150339 28855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:24.150372 28855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:24.151937 28855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:33333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:24.152244 28855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:24.152495 28855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:24.157704 28861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:24.157707 28862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:24.157768 28864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:24.158349 28855 server_base.cc:1034] running on GCE node
I20260805 22:42:24.158681 28855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:24.159015 28855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:24.160238 28855 hybrid_clock.cc:648] HybridClock initialized: now 1785969744160198 us; error 57 us; skew 500 ppm
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:24.163228 28855 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:24.163841 28855 webserver.cc:460] Webserver started at http://127.23.197.126:33217/ using document root <none> and password file <none>
I20260805 22:42:24.164113 28855 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:24.164214 28855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:24.164387 28855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:24.165701 28855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "4a729c6ef29347739d35e7d3709ad78c"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "e6ef7facf5c8295e0d147990353b2518"
server_key_iv: "0d975b793bf31a9065814a56d6ad2240"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.166131 28855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "4a729c6ef29347739d35e7d3709ad78c"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "e6ef7facf5c8295e0d147990353b2518"
server_key_iv: "0d975b793bf31a9065814a56d6ad2240"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.167692 28855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:42:24.168602 28874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.168860 28855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:24.168977 28855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "4a729c6ef29347739d35e7d3709ad78c"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "e6ef7facf5c8295e0d147990353b2518"
server_key_iv: "0d975b793bf31a9065814a56d6ad2240"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.169059 28855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:24.199895 28855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:24.202030 28855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:24.202234 28855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:24.206553 28855 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:33333
I20260805 22:42:24.206590 28959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:33333 every 8 connection(s)
I20260805 22:42:24.206975 28855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:24.208003 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 28855
I20260805 22:42:24.208099 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:24.208309 24341 external_mini_cluster.cc:1428] Setting key ccc55586dfe20374273e53ba1f110f32
I20260805 22:42:24.208542 28960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:24.212524 28960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c: Bootstrap starting.
I20260805 22:42:24.213356 28960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:24.213723 28960 log.cc:824] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:24.214560 28960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c: No bootstrap required, opened a new log
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:24.216581 28960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a729c6ef29347739d35e7d3709ad78c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33333 } }
I20260805 22:42:24.216763 28960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:24.216828 28960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a729c6ef29347739d35e7d3709ad78c, State: Initialized, Role: FOLLOWER
I20260805 22:42:24.217010 28960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a729c6ef29347739d35e7d3709ad78c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33333 } }
I20260805 22:42:24.217121 28960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:24.217166 28960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:24.217217 28960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:24.217783 28960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a729c6ef29347739d35e7d3709ad78c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33333 } }
I20260805 22:42:24.217911 28960 leader_election.cc:302] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a729c6ef29347739d35e7d3709ad78c; no voters:
I20260805 22:42:24.218108 28960 leader_election.cc:288] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:24.218173 28965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:24.218354 28965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [term 1 LEADER]: Becoming Leader. State: Replica: 4a729c6ef29347739d35e7d3709ad78c, State: Running, Role: LEADER
I20260805 22:42:24.218448 28965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a729c6ef29347739d35e7d3709ad78c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33333 } }
I20260805 22:42:24.218453 28960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:24.218895 28967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a729c6ef29347739d35e7d3709ad78c. Latest consensus state: current_term: 1 leader_uuid: "4a729c6ef29347739d35e7d3709ad78c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a729c6ef29347739d35e7d3709ad78c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33333 } } }
I20260805 22:42:24.219125 28967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:24.219062 28966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a729c6ef29347739d35e7d3709ad78c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a729c6ef29347739d35e7d3709ad78c" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33333 } } }
I20260805 22:42:24.219414 28971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:24.219424 28966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:24.220285 28971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:24.221095 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:24.208929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.23.197.126:33333)
0805 22:42:24.209143 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:24.209156 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:24.209196 (+ 40us) server_negotiation.cc:407] Connection header received
0805 22:42:24.209945 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:24.209956 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:24.210026 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:24.210215 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:24.210824 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.212364 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.213280 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.213988 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.214155 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:24.216620 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:24.216624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:24.216624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:24.216635 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:24.218857 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.219236 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.219240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.219240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.219358 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.219608 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.219609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.219609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.219931 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:24.220533 (+ 602us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:24.220709 (+ 176us) server_negotiation.cc:299] Negotiation successful
0805 22:42:24.220891 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260805 22:42:24.222126 28971 catalog_manager.cc:1347] Generated new cluster ID: 909c2f5136da4b06a0def6eb2f89b3b8
I20260805 22:42:24.222178 28971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:24.232888 28971 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:24.233484 28971 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:24.240190 28971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a729c6ef29347739d35e7d3709ad78c: Generated new TSK 0
I20260805 22:42:24.240406 28971 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:24.272226 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33333
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:24.377818 28996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:24.378057 28996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:24.378090 28996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:24.379302 28996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:24.379344 28996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:24.379372 28996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:24.380954 28996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:24.381165 28996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:24.381369 28996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:24.387053 29005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:24.386997 29003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:24.387012 29008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:24.387322 28996 server_base.cc:1034] running on GCE node
I20260805 22:42:24.387565 28996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:24.387825 28996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:24.389045 28996 hybrid_clock.cc:648] HybridClock initialized: now 1785969744389036 us; error 42 us; skew 500 ppm
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:24.391945 28996 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:24.392460 28996 webserver.cc:460] Webserver started at http://127.23.197.65:45721/ using document root <none> and password file <none>
I20260805 22:42:24.392715 28996 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:24.392874 28996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:24.393040 28996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:24.394177 28996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "14a3f891d3ca44a7a4a0a0f5cec6db9f"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "3d45e4a16b17e618ba4072007ac7275e"
server_key_iv: "385d5d62c234a6555aed1c811c02fc66"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.394577 28996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "14a3f891d3ca44a7a4a0a0f5cec6db9f"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "3d45e4a16b17e618ba4072007ac7275e"
server_key_iv: "385d5d62c234a6555aed1c811c02fc66"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.396054 28996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:24.396999 29016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.397310 28996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:24.397428 28996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "14a3f891d3ca44a7a4a0a0f5cec6db9f"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "3d45e4a16b17e618ba4072007ac7275e"
server_key_iv: "385d5d62c234a6555aed1c811c02fc66"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.397506 28996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:24.424813 28996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:24.427372 28996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:24.427493 28996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:24.427695 28996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:24.427969 28996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:24.428000 28996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.428022 28996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:24.428035 28996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.434199 28996 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:37113
I20260805 22:42:24.434310 29194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:37113 every 8 connection(s)
I20260805 22:42:24.434626 28996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:24.440941 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 28996
I20260805 22:42:24.441079 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:24.441305 24341 external_mini_cluster.cc:1428] Setting key 176fce8b413dcc32906a582a50ed0d74
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:24.445899 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:24.435433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:48051 (local address 127.23.197.126:33333)
0805 22:42:24.435533 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:24.435536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:24.435718 (+ 182us) server_negotiation.cc:407] Connection header received
0805 22:42:24.436270 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:24.436272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:24.436285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:24.436337 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:24.437422 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.437937 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.439355 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.439685 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.439777 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:24.442521 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:24.442523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:24.442523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:24.442532 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:24.444247 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.444810 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.444811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.444812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.444879 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.445177 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.445177 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.445178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.445296 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:24.445354 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:24.445752 (+ 398us) server_negotiation.cc:299] Negotiation successful
0805 22:42:24.445832 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 22:42:24.446132 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:24.435346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33333 (local address 127.23.197.65:48051)
0805 22:42:24.435569 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:24.435620 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:24.436148 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:24.436414 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:24.436418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:24.437107 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:24.437313 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:24.437318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.438063 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.438065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:24.439263 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:24.439266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.439801 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.439802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:24.439953 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:24.440602 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:24.440612 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:24.442426 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:24.444340 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:24.444348 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:24.444351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:24.444687 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:24.444951 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:24.444952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:24.444952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:24.445100 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:24.445371 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:24.445378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:24.445547 (+ 169us) client_negotiation.cc:769] Sending connection context
0805 22:42:24.445668 (+ 121us) client_negotiation.cc:240] Negotiation successful
0805 22:42:24.445854 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":95,"threads_started":1}
I20260805 22:42:24.446640 29196 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33333
I20260805 22:42:24.446765 29196 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:24.446976 29196 heartbeater.cc:511] Master 127.23.197.126:33333 requested a full tablet report, sending...
I20260805 22:42:24.447618 28899 ts_manager.cc:194] Registered new tserver with Master: 14a3f891d3ca44a7a4a0a0f5cec6db9f (127.23.197.65:37113)
I20260805 22:42:24.448278 28899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:48051
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:24.469908 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33333
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:24.554338 29204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:24.554580 29204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:24.554618 29204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:24.555758 29204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:24.555830 29204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:24.555876 29204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:24.557349 29204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:24.557627 29204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:24.557880 29204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:24.563091 29210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:24.563119 29214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:24.563091 29211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:24.563469 29204 server_base.cc:1034] running on GCE node
I20260805 22:42:24.563715 29204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:24.563975 29204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:24.565142 29204 hybrid_clock.cc:648] HybridClock initialized: now 1785969744565128 us; error 27 us; skew 500 ppm
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:24.567751 29204 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:24.568282 29204 webserver.cc:460] Webserver started at http://127.23.197.66:42619/ using document root <none> and password file <none>
I20260805 22:42:24.568554 29204 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:24.568626 29204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:24.568813 29204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:24.569972 29204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "a17127d5b2454017a684759c6a37f028"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "04320f750cb8125325ff4cbd145bab45"
server_key_iv: "442bdaf83bd8486e22b36b718512e493"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.570356 29204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "a17127d5b2454017a684759c6a37f028"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "04320f750cb8125325ff4cbd145bab45"
server_key_iv: "442bdaf83bd8486e22b36b718512e493"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.571825 29204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:24.572677 29225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.572868 29204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:24.572979 29204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "a17127d5b2454017a684759c6a37f028"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "04320f750cb8125325ff4cbd145bab45"
server_key_iv: "442bdaf83bd8486e22b36b718512e493"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.573056 29204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:24.582435 29204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:24.584842 29204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:24.585005 29204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:24.585247 29204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:24.585620 29204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:24.585677 29204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.585709 29204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:24.585740 29204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.591766 29204 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:43275
I20260805 22:42:24.591818 29393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:43275 every 8 connection(s)
I20260805 22:42:24.592125 29204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:24.595715 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 29204
I20260805 22:42:24.595911 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:24.596122 24341 external_mini_cluster.cc:1428] Setting key 2e18255f269238790fd566973e71816f
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:24.603102 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:24.592940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:57403 (local address 127.23.197.126:33333)
0805 22:42:24.593020 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:24.593023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:24.593294 (+ 271us) server_negotiation.cc:407] Connection header received
0805 22:42:24.593783 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:24.593785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:24.593797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:24.593861 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:24.594806 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.595330 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.596585 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.596913 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.597010 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:24.599707 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:24.599709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:24.599709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:24.599719 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:24.601609 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.602020 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.602022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.602022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.602104 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.602378 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.602379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.602379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.602545 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:24.602626 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:24.602902 (+ 276us) server_negotiation.cc:299] Negotiation successful
0805 22:42:24.602998 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 22:42:24.603176 29397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:24.592947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33333 (local address 127.23.197.66:57403)
0805 22:42:24.593133 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:24.593169 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:24.593669 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:24.593912 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:24.593915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:24.594499 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:24.594696 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:24.594700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.595427 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.595428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:24.596514 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:24.596517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.597021 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.597022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:24.597148 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:24.597746 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:24.597755 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:24.599606 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:24.601697 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:24.601705 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:24.601708 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:24.601939 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:24.602183 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:24.602184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:24.602184 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:24.602305 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:24.602620 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:24.602626 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:24.602760 (+ 134us) client_negotiation.cc:769] Sending connection context
0805 22:42:24.602852 (+ 92us) client_negotiation.cc:240] Negotiation successful
0805 22:42:24.602980 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260805 22:42:24.603809 29394 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33333
I20260805 22:42:24.603904 29394 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:24.604099 29394 heartbeater.cc:511] Master 127.23.197.126:33333 requested a full tablet report, sending...
I20260805 22:42:24.604557 28899 ts_manager.cc:194] Registered new tserver with Master: a17127d5b2454017a684759c6a37f028 (127.23.197.66:43275)
I20260805 22:42:24.605168 28899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:57403
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:24.625881 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33333
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:24.712990 29401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:24.713219 29401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:24.713244 29401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:24.714392 29401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:24.714428 29401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:24.714453 29401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:24.715909 29401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:24.716112 29401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:24.716300 29401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:24.723057 29410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:24.723047 29407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:24.723199 29401 server_base.cc:1034] running on GCE node
W20260805 22:42:24.723049 29408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:24.723505 29401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:24.723781 29401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:24.725003 29401 hybrid_clock.cc:648] HybridClock initialized: now 1785969744724976 us; error 36 us; skew 500 ppm
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:24.727519 29401 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:24.728049 29401 webserver.cc:460] Webserver started at http://127.23.197.67:46333/ using document root <none> and password file <none>
I20260805 22:42:24.728299 29401 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:24.728372 29401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:24.728533 29401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:24.729686 29401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "9f720837bcee4ca09ed613491b686a27"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "9739ac691c83b71230665186fb56aa4c"
server_key_iv: "6c8393e6d85433c17cc79d8c267eed92"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.730127 29401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "9f720837bcee4ca09ed613491b686a27"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "9739ac691c83b71230665186fb56aa4c"
server_key_iv: "6c8393e6d85433c17cc79d8c267eed92"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.731549 29401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:24.732309 29417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.732481 29401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:24.732589 29401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "9f720837bcee4ca09ed613491b686a27"
format_stamp: "Formatted at 2026-08-05 22:42:24 on dist-test-slave-g170"
server_key: "9739ac691c83b71230665186fb56aa4c"
server_key_iv: "6c8393e6d85433c17cc79d8c267eed92"
server_key_version: "encryptionkey@0"
I20260805 22:42:24.732663 29401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:24.746470 29401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:24.748346 29401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:24.748519 29401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:24.748816 29401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:24.749166 29401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:24.749222 29401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.749264 29401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:24.749290 29401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:24.755822 29401 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:43511
I20260805 22:42:24.755867 29588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:43511 every 8 connection(s)
I20260805 22:42:24.756249 29401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:24.761349 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 29401
I20260805 22:42:24.761477 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:24.761694 24341 external_mini_cluster.cc:1428] Setting key bd13864336a99d381a4c7bacd17c8066
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:24.766368 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:24.757162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:38045 (local address 127.23.197.126:33333)
0805 22:42:24.757247 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:24.757249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:24.757445 (+ 196us) server_negotiation.cc:407] Connection header received
0805 22:42:24.758074 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:24.758076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:24.758090 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:24.758172 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:24.759205 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.759725 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.760807 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.761028 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.761107 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:24.763672 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:24.763673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:24.763674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:24.763681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:24.765093 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.765514 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.765515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.765516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.765570 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.765797 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.765798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.765798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.765911 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:24.765964 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:24.766231 (+ 267us) server_negotiation.cc:299] Negotiation successful
0805 22:42:24.766301 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 22:42:24.766587 29592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:24.757138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33333 (local address 127.23.197.67:38045)
0805 22:42:24.757306 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:24.757345 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:24.757949 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:24.758253 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:24.758257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:24.758883 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:24.759088 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:24.759092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.759828 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.759830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:24.760671 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:24.760673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.761131 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.761132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:24.761268 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:24.761863 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:24.761872 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:24.763580 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:24.765173 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:24.765181 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:24.765183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:24.765440 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:24.765630 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:24.765631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:24.765631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:24.765736 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:24.765971 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:24.765976 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:24.766110 (+ 134us) client_negotiation.cc:769] Sending connection context
0805 22:42:24.766198 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 22:42:24.766350 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260805 22:42:24.767118 29589 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33333
I20260805 22:42:24.767220 29589 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:24.767385 29589 heartbeater.cc:511] Master 127.23.197.126:33333 requested a full tablet report, sending...
I20260805 22:42:24.767740 28899 ts_manager.cc:194] Registered new tserver with Master: 9f720837bcee4ca09ed613491b686a27 (127.23.197.67:43511)
I20260805 22:42:24.768076 28899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:38045
I20260805 22:42:24.773854 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[28835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969744, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:24.800406 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:24.792208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.23.197.126:33333)
0805 22:42:24.792303 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:24.792305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:24.792402 (+ 97us) server_negotiation.cc:407] Connection header received
0805 22:42:24.792535 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:24.792536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:24.792546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:24.792633 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:24.793361 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.793875 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.794726 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:24.794923 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:24.795000 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:24.797507 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:24.797509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:24.797509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:24.797516 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:24.799018 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.799411 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.799412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.799412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.799469 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:24.799648 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:24.799649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:24.799649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:24.799846 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:24.799928 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:24.800185 (+ 257us) server_negotiation.cc:299] Negotiation successful
0805 22:42:24.800260 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260805 22:42:24.800781 28899 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43694
I20260805 22:42:24.801401 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 28996
I20260805 22:42:24.820174 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 29204
I20260805 22:42:24.831300 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 29401
I20260805 22:42:24.850944 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 28855
2026-08-05T22:42:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2206 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[29607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[29607](info): set up 2 sockets
Aug 05 22:42:24 dist-test-slave-g170 krb5kdc[29607](info): commencing operation
krb5kdc: starting...
W20260805 22:42:26.085824 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:26.219410 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41175
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:41175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:26.308488 29625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:26.308758 29625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:26.308820 29625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:26.310009 29625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:26.310082 29625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:26.310108 29625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:26.310129 29625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:26.310163 29625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:26.311717 29625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:41175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:26.312007 29625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:26.312250 29625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:26.317667 29636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:26.317700 29634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:26.317886 29625 server_base.cc:1034] running on GCE node
W20260805 22:42:26.317698 29633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:26.318181 29625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:26.318504 29625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:26.319739 29625 hybrid_clock.cc:648] HybridClock initialized: now 1785969746319737 us; error 44 us; skew 500 ppm
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:26.326615 29625 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:26.327142 29625 webserver.cc:460] Webserver started at http://127.23.197.126:37399/ using document root <none> and password file <none>
I20260805 22:42:26.327354 29625 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:26.327399 29625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:26.327495 29625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:26.328651 29625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "58bd6bc6711d4d83ae6727191b2a7415"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "66a02c2879b800b4f0ba3b80d188845f"
server_key_iv: "789aeedc3da467e6b892e07d67e3126d"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.329111 29625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "58bd6bc6711d4d83ae6727191b2a7415"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "66a02c2879b800b4f0ba3b80d188845f"
server_key_iv: "789aeedc3da467e6b892e07d67e3126d"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.330674 29625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:26.331667 29652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.331957 29625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:26.332098 29625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "58bd6bc6711d4d83ae6727191b2a7415"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "66a02c2879b800b4f0ba3b80d188845f"
server_key_iv: "789aeedc3da467e6b892e07d67e3126d"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.332188 29625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:26.357393 29625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:26.360009 29625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:26.360193 29625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:26.364804 29625 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:41175
I20260805 22:42:26.364879 29718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:41175 every 8 connection(s)
I20260805 22:42:26.365399 29625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:26.366156 29719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:26.368530 29719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415: Bootstrap starting.
I20260805 22:42:26.369124 29719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:26.369397 29719 log.cc:824] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:26.370122 29719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415: No bootstrap required, opened a new log
I20260805 22:42:26.371745 29719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bd6bc6711d4d83ae6727191b2a7415" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41175 } }
I20260805 22:42:26.371887 29719 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:26.371910 29719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58bd6bc6711d4d83ae6727191b2a7415, State: Initialized, Role: FOLLOWER
I20260805 22:42:26.372076 29719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bd6bc6711d4d83ae6727191b2a7415" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41175 } }
I20260805 22:42:26.372175 29719 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:26.372200 29719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:26.372242 29719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:26.373147 29719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bd6bc6711d4d83ae6727191b2a7415" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41175 } }
I20260805 22:42:26.373246 29719 leader_election.cc:302] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58bd6bc6711d4d83ae6727191b2a7415; no voters:
I20260805 22:42:26.373394 29719 leader_election.cc:288] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:26.373479 29725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:26.373658 29719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:26.373675 29725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [term 1 LEADER]: Becoming Leader. State: Replica: 58bd6bc6711d4d83ae6727191b2a7415, State: Running, Role: LEADER
I20260805 22:42:26.373886 29725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bd6bc6711d4d83ae6727191b2a7415" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41175 } }
I20260805 22:42:26.374296 29728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58bd6bc6711d4d83ae6727191b2a7415. Latest consensus state: current_term: 1 leader_uuid: "58bd6bc6711d4d83ae6727191b2a7415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bd6bc6711d4d83ae6727191b2a7415" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41175 } } }
I20260805 22:42:26.374317 29727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58bd6bc6711d4d83ae6727191b2a7415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58bd6bc6711d4d83ae6727191b2a7415" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41175 } } }
I20260805 22:42:26.374476 29727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:26.374701 29747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:26.374936 29728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:26.375303 29747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:26.375469 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 29625
I20260805 22:42:26.375577 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:26.375775 24341 external_mini_cluster.cc:1428] Setting key 4c8a060253922a9eda9011aafba2ae75
I20260805 22:42:26.377066 29747 catalog_manager.cc:1347] Generated new cluster ID: f9af12518b704dc2a277d65f15f3de5e
I20260805 22:42:26.377123 29747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:26.388021 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:26.376500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39678 (local address 127.23.197.126:41175)
0805 22:42:26.376706 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:26.376718 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:26.376815 (+ 97us) server_negotiation.cc:407] Connection header received
0805 22:42:26.377000 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:26.377008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:26.377078 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:26.377226 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:26.378056 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.379447 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.380545 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.381182 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.381285 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:26.384190 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:26.384195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:26.384195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:26.384208 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:26.386223 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.386617 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.386620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.386621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.386756 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.386961 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.386962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.386962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.387274 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:26.387439 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:26.387513 (+ 74us) server_negotiation.cc:299] Negotiation successful
0805 22:42:26.387767 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":81,"threads_started":1}
I20260805 22:42:26.392407 29747 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:26.392987 29747 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:26.401865 29747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58bd6bc6711d4d83ae6727191b2a7415: Generated new TSK 0
I20260805 22:42:26.402062 29747 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:26.436363 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41175
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:26.524875 29768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:26.525082 29768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:26.525107 29768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:26.526192 29768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:26.526228 29768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:26.526253 29768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:26.527734 29768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:26.527932 29768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:26.528115 29768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:26.533403 29774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:26.533519 29777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:26.533600 29768 server_base.cc:1034] running on GCE node
W20260805 22:42:26.533746 29775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:26.533993 29768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:26.534255 29768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:26.535422 29768 hybrid_clock.cc:648] HybridClock initialized: now 1785969746535417 us; error 40 us; skew 500 ppm
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:26.538172 29768 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:26.538635 29768 webserver.cc:460] Webserver started at http://127.23.197.65:44911/ using document root <none> and password file <none>
I20260805 22:42:26.538880 29768 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:26.538951 29768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:26.539109 29768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:26.540208 29768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "46f893da7a2740719c461aa7fa3ae2b2"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "e32f672938dc7ffd79b8519fd961d7e6"
server_key_iv: "42a0aa1fe1d3d91f5bcc9fe20b408bc9"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.540608 29768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "46f893da7a2740719c461aa7fa3ae2b2"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "e32f672938dc7ffd79b8519fd961d7e6"
server_key_iv: "42a0aa1fe1d3d91f5bcc9fe20b408bc9"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.542094 29768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:26.542965 29785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.543138 29768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:26.543258 29768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "46f893da7a2740719c461aa7fa3ae2b2"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "e32f672938dc7ffd79b8519fd961d7e6"
server_key_iv: "42a0aa1fe1d3d91f5bcc9fe20b408bc9"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.543339 29768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:26.553828 29768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:26.555863 29768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:26.556047 29768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:26.556308 29768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:26.556638 29768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:26.556694 29768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.556727 29768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:26.556841 29768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.562902 29768 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:35987
I20260805 22:42:26.562954 29946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:35987 every 8 connection(s)
I20260805 22:42:26.563287 29768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:26.572515 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 29768
I20260805 22:42:26.572635 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:26.572888 24341 external_mini_cluster.cc:1428] Setting key c9054d0312f655d753927bb5f34bfdcc
I20260805 22:42:26.573195 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:26.564130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:40699 (local address 127.23.197.126:41175)
0805 22:42:26.564199 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:26.564201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:26.564382 (+ 181us) server_negotiation.cc:407] Connection header received
0805 22:42:26.565333 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:26.565334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:26.565347 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:26.565409 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:26.566057 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.566592 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.567571 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.567776 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.567853 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:26.570059 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:26.570061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:26.570062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:26.570070 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:26.571523 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.571963 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.571964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.571965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.572030 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.572308 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.572309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.572309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.572468 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:26.572549 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:26.573002 (+ 453us) server_negotiation.cc:299] Negotiation successful
0805 22:42:26.573103 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 22:42:26.573392 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:26.564046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41175 (local address 127.23.197.65:40699)
0805 22:42:26.564244 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:26.564288 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:26.565232 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:26.565472 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:26.565475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:26.565750 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:26.565964 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:26.565969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.566681 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.566682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:26.567483 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:26.567485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.567874 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.567875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:26.567984 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:26.568478 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:26.568488 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:26.569940 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:26.571633 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:26.571641 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:26.571643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:26.571882 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:26.572102 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:26.572103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:26.572103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:26.572227 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:26.572574 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:26.572581 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:26.572848 (+ 267us) client_negotiation.cc:769] Sending connection context
0805 22:42:26.572974 (+ 126us) client_negotiation.cc:240] Negotiation successful
0805 22:42:26.573138 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260805 22:42:26.574117 29947 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41175
I20260805 22:42:26.574211 29947 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:26.574514 29947 heartbeater.cc:511] Master 127.23.197.126:41175 requested a full tablet report, sending...
I20260805 22:42:26.575084 29675 ts_manager.cc:194] Registered new tserver with Master: 46f893da7a2740719c461aa7fa3ae2b2 (127.23.197.65:35987)
I20260805 22:42:26.575937 29675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:40699
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:26.600232 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41175
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:26.687175 29952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:26.687386 29952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:26.687410 29952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:26.688493 29952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:26.688530 29952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:26.688556 29952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:26.690029 29952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:26.690253 29952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:26.690436 29952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:26.695788 29962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:26.695811 29952 server_base.cc:1034] running on GCE node
W20260805 22:42:26.695778 29959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:26.695775 29958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:26.696271 29952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:26.696486 29952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:26.697691 29952 hybrid_clock.cc:648] HybridClock initialized: now 1785969746697668 us; error 50 us; skew 500 ppm
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:26.700361 29952 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:26.701071 29952 webserver.cc:460] Webserver started at http://127.23.197.66:43685/ using document root <none> and password file <none>
I20260805 22:42:26.701351 29952 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:26.701426 29952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:26.701582 29952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:26.702677 29952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "c79ef4a4382344f3b4056b4f8671ff78"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "85bbb60d8550ccfbc71abf4988555ccb"
server_key_iv: "837ef9e9623484c4f0a0fb38d715731c"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.703052 29952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "c79ef4a4382344f3b4056b4f8671ff78"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "85bbb60d8550ccfbc71abf4988555ccb"
server_key_iv: "837ef9e9623484c4f0a0fb38d715731c"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.704444 29952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:26.705284 29971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.705456 29952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.705567 29952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "c79ef4a4382344f3b4056b4f8671ff78"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "85bbb60d8550ccfbc71abf4988555ccb"
server_key_iv: "837ef9e9623484c4f0a0fb38d715731c"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.705641 29952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:26.723137 29952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:26.725492 29952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:26.725672 29952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:26.725943 29952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:26.726290 29952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:26.726347 29952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.726394 29952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:26.726420 29952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.733541 29952 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:40343
I20260805 22:42:26.733584 30131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:40343 every 8 connection(s)
I20260805 22:42:26.733938 29952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:26.736393 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 29952
I20260805 22:42:26.736514 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:26.736711 24341 external_mini_cluster.cc:1428] Setting key af919c27af7ae6d1ed309563a27f76e1
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:26.744177 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:26.734819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:48221 (local address 127.23.197.126:41175)
0805 22:42:26.734881 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:26.734883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:26.735082 (+ 199us) server_negotiation.cc:407] Connection header received
0805 22:42:26.736019 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:26.736020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:26.736056 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:26.736119 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:26.737080 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.737587 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.738627 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.738828 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.738911 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:26.741202 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:26.741204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:26.741205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:26.741213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:26.742708 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.743124 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.743125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.743125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.743190 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.743453 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.743454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.743454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.743574 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:26.743656 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:26.743984 (+ 328us) server_negotiation.cc:299] Negotiation successful
0805 22:42:26.744085 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260805 22:42:26.744377 30134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:26.734700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41175 (local address 127.23.197.66:48221)
0805 22:42:26.734931 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:26.734977 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:26.735929 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:26.736205 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:26.736210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:26.736625 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:26.736966 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:26.736972 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.737700 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.737701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:26.738534 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:26.738536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.738929 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.738930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:26.739057 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:26.739651 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:26.739670 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:26.741099 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:26.742797 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:26.742804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:26.742806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:26.743044 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:26.743265 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:26.743266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:26.743266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:26.743378 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:26.743658 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:26.743665 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:26.743839 (+ 174us) client_negotiation.cc:769] Sending connection context
0805 22:42:26.743949 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 22:42:26.744115 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":120,"threads_started":1}
I20260805 22:42:26.744908 30132 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41175
I20260805 22:42:26.744994 30132 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:26.745198 30132 heartbeater.cc:511] Master 127.23.197.126:41175 requested a full tablet report, sending...
I20260805 22:42:26.745645 29675 ts_manager.cc:194] Registered new tserver with Master: c79ef4a4382344f3b4056b4f8671ff78 (127.23.197.66:40343)
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
I20260805 22:42:26.746109 29675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:48221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:26.768689 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41175
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:26.854390 30139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:26.854656 30139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:26.854696 30139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:26.855911 30139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:26.855979 30139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:26.856024 30139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:26.857616 30139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:26.857903 30139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:26.858148 30139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:26.863288 30148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:26.863327 30154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:26.863381 30152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:26.863521 30139 server_base.cc:1034] running on GCE node
I20260805 22:42:26.863787 30139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:26.864071 30139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:26.865236 30139 hybrid_clock.cc:648] HybridClock initialized: now 1785969746865219 us; error 36 us; skew 500 ppm
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:26.867897 30139 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:26.868356 30139 webserver.cc:460] Webserver started at http://127.23.197.67:38257/ using document root <none> and password file <none>
I20260805 22:42:26.868625 30139 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:26.868752 30139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:26.868968 30139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:26.870123 30139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "c2b00590e5564597a96428abd38580da"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "9b9ef869d7ac6088b7dd0024513ddd9c"
server_key_iv: "0676d7661ba9ba38b3e77d216b845eab"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.870507 30139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "c2b00590e5564597a96428abd38580da"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "9b9ef869d7ac6088b7dd0024513ddd9c"
server_key_iv: "0676d7661ba9ba38b3e77d216b845eab"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.871868 30139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:26.872635 30162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.872893 30139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:26.872998 30139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "c2b00590e5564597a96428abd38580da"
format_stamp: "Formatted at 2026-08-05 22:42:26 on dist-test-slave-g170"
server_key: "9b9ef869d7ac6088b7dd0024513ddd9c"
server_key_iv: "0676d7661ba9ba38b3e77d216b845eab"
server_key_version: "encryptionkey@0"
I20260805 22:42:26.873080 30139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:26.894292 30139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:26.896989 30139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:26.897161 30139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:26.897447 30139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:26.897836 30139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:26.897903 30139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.897949 30139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:26.897993 30139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:26.904549 30139 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:36973
I20260805 22:42:26.904597 30325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:36973 every 8 connection(s)
I20260805 22:42:26.905009 30139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:26.914889 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:26.905826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:47715 (local address 127.23.197.126:41175)
0805 22:42:26.905898 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:26.905900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:26.906088 (+ 188us) server_negotiation.cc:407] Connection header received
0805 22:42:26.907016 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:26.907017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:26.907027 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:26.907092 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:26.907728 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.908310 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.909662 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.909859 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.909927 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:26.912132 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:26.912133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:26.912133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:26.912140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:26.913543 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.913926 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.913927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.913928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.913981 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.914232 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.914233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.914233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.914338 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:26.914391 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:26.914635 (+ 244us) server_negotiation.cc:299] Negotiation successful
0805 22:42:26.914740 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 22:42:26.914963 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:26.905778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41175 (local address 127.23.197.67:47715)
0805 22:42:26.905949 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:26.905995 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:26.906925 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:26.907147 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:26.907151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:26.907418 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:26.907612 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:26.907616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.908466 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.908467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:26.909493 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:26.909496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.909945 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.909946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:26.910064 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:26.910624 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:26.910634 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:26.912037 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:26.913621 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:26.913627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:26.913629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:26.913859 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:26.914048 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:26.914049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:26.914050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:26.914171 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:26.914397 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:26.914402 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:26.914517 (+ 115us) client_negotiation.cc:769] Sending connection context
0805 22:42:26.914601 (+ 84us) client_negotiation.cc:240] Negotiation successful
0805 22:42:26.914767 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260805 22:42:26.914970 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 30139
I20260805 22:42:26.915064 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:26.915297 24341 external_mini_cluster.cc:1428] Setting key b1b4d243fd864aa29df72a0e7b17f7b6
I20260805 22:42:26.915489 30326 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41175
I20260805 22:42:26.915616 30326 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:26.915810 30326 heartbeater.cc:511] Master 127.23.197.126:41175 requested a full tablet report, sending...
I20260805 22:42:26.916190 29675 ts_manager.cc:194] Registered new tserver with Master: c2b00590e5564597a96428abd38580da (127.23.197.67:36973)
I20260805 22:42:26.916599 29675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:47715
I20260805 22:42:26.927456 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:26.954015 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:26.945300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54172 (local address 127.23.197.65:35987)
0805 22:42:26.945542 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:26.945548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:26.945559 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:26.945585 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:26.945587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:26.945601 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:26.945684 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:26.946410 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.947242 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.948046 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.948471 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.948547 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:26.950913 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:26.950922 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:26.950923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:26.950930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:26.952637 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.953079 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.953082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.953083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.953144 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.953403 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.953404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.953405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.953569 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:26.953695 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:26.953823 (+ 128us) server_negotiation.cc:299] Negotiation successful
0805 22:42:26.953891 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":132,"threads_started":1}
Aug 05 22:42:26 dist-test-slave-g170 krb5kdc[29607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:26.962211 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:26.955484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39694 (local address 127.23.197.126:41175)
0805 22:42:26.955540 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:26.955542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:26.955550 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:26.955591 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:26.955592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:26.955601 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:26.955647 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:26.956119 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.956599 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.957344 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:26.957524 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:26.957578 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:26.959836 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:26.959837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:26.959837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:26.959844 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:26.961258 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.961563 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.961563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.961564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.961616 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:26.961780 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:26.961781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:26.961781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:26.961924 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:26.962001 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:26.962085 (+ 84us) server_negotiation.cc:299] Negotiation successful
0805 22:42:26.962144 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260805 22:42:26.962497 29675 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39694
I20260805 22:42:26.963004 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 29768
I20260805 22:42:26.989394 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 29952
I20260805 22:42:26.996235 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 30139
I20260805 22:42:27.008035 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 29625
2026-08-05T22:42:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2159 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:27 dist-test-slave-g170 krb5kdc[30343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:27 dist-test-slave-g170 krb5kdc[30343](info): set up 2 sockets
Aug 05 22:42:27 dist-test-slave-g170 krb5kdc[30343](info): commencing operation
krb5kdc: starting...
W20260805 22:42:28.258447 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:28 dist-test-slave-g170 krb5kdc[30343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:28.397570 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36331
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:36331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:28.482647 30360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:28.482915 30360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:28.482955 30360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:28.484158 30360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:28.484226 30360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:28.484254 30360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:28.484277 30360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:28.484303 30360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:28.485939 30360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:36331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:28.486234 30360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:28.486469 30360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 22:42:28.491664 30360 server_base.cc:1034] running on GCE node
W20260805 22:42:28.491607 30370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:28.491624 30374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:28.491608 30369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:28.492115 30360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:28.492445 30360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:28.493676 30360 hybrid_clock.cc:648] HybridClock initialized: now 1785969748493662 us; error 28 us; skew 500 ppm
Aug 05 22:42:28 dist-test-slave-g170 krb5kdc[30343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969748, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:28.496331 30360 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:28.496735 30360 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:28.496848 30360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:28.496956 30360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:28.498018 30360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "01b4a79c5ce943c98b8a10e463dae543"
format_stamp: "Formatted at 2026-08-05 22:42:28 on dist-test-slave-g170"
server_key: "ac98c7a9c5e837b00c97686cda866ad1"
server_key_iv: "aeef22e950b61cd3640bd25990864168"
server_key_version: "encryptionkey@0"
I20260805 22:42:28.498343 30360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "01b4a79c5ce943c98b8a10e463dae543"
format_stamp: "Formatted at 2026-08-05 22:42:28 on dist-test-slave-g170"
server_key: "ac98c7a9c5e837b00c97686cda866ad1"
server_key_iv: "aeef22e950b61cd3640bd25990864168"
server_key_version: "encryptionkey@0"
I20260805 22:42:28.499683 30360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:28.500452 30388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:28.500659 30360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:28.500751 30360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "01b4a79c5ce943c98b8a10e463dae543"
format_stamp: "Formatted at 2026-08-05 22:42:28 on dist-test-slave-g170"
server_key: "ac98c7a9c5e837b00c97686cda866ad1"
server_key_iv: "aeef22e950b61cd3640bd25990864168"
server_key_version: "encryptionkey@0"
I20260805 22:42:28.500864 30360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:28.511732 30360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:28.513600 30360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:28.513715 30360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:28.517618 30360 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:36331
I20260805 22:42:28.517661 30470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:36331 every 8 connection(s)
I20260805 22:42:28.517966 30360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:28.518929 30472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:28.521322 30472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543: Bootstrap starting.
I20260805 22:42:28.521927 30472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:28.522226 30472 log.cc:824] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:28.522933 30472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543: No bootstrap required, opened a new log
I20260805 22:42:28.523102 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 30360
I20260805 22:42:28.523252 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:28.523469 24341 external_mini_cluster.cc:1428] Setting key 86b2ed83efc21d9a26bd4246f0ac40fb
I20260805 22:42:28.525897 30472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01b4a79c5ce943c98b8a10e463dae543" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36331 } }
I20260805 22:42:28.526047 30472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:28.526094 30472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01b4a79c5ce943c98b8a10e463dae543, State: Initialized, Role: FOLLOWER
I20260805 22:42:28.526221 30472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01b4a79c5ce943c98b8a10e463dae543" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36331 } }
I20260805 22:42:28.526314 30472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:28.526357 30472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:28.526404 30472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:28.527004 30472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01b4a79c5ce943c98b8a10e463dae543" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36331 } }
I20260805 22:42:28.527117 30472 leader_election.cc:302] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01b4a79c5ce943c98b8a10e463dae543; no voters:
I20260805 22:42:28.527283 30472 leader_election.cc:288] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:28.527357 30482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:28.527508 30482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [term 1 LEADER]: Becoming Leader. State: Replica: 01b4a79c5ce943c98b8a10e463dae543, State: Running, Role: LEADER
I20260805 22:42:28.527534 30472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:28.528638 30482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01b4a79c5ce943c98b8a10e463dae543" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36331 } }
I20260805 22:42:28.529867 30483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01b4a79c5ce943c98b8a10e463dae543. Latest consensus state: current_term: 1 leader_uuid: "01b4a79c5ce943c98b8a10e463dae543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01b4a79c5ce943c98b8a10e463dae543" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36331 } } }
I20260805 22:42:28.530031 30483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:28.530290 30500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:28.529912 30484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01b4a79c5ce943c98b8a10e463dae543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01b4a79c5ce943c98b8a10e463dae543" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36331 } } }
I20260805 22:42:28.530445 30484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:28.530898 30500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 05 22:42:28 dist-test-slave-g170 krb5kdc[30343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:28.533159 30500 catalog_manager.cc:1347] Generated new cluster ID: d80879a91c9e4587944603d02cd8c0e2
I20260805 22:42:28.533217 30500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:28.535424 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:28.523980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.23.197.126:36331)
0805 22:42:28.524155 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:28.524163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:28.524188 (+ 25us) server_negotiation.cc:407] Connection header received
0805 22:42:28.524822 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:28.524831 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:28.524881 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:28.524997 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:28.525841 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.526866 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.527961 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.528679 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.528818 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:28.531851 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:28.531855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:28.531856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:28.531868 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:28.533820 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:28.534218 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:28.534221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:28.534222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:28.534317 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:28.534528 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:28.534529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:28.534529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:28.534777 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:28.534941 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:28.535054 (+ 113us) server_negotiation.cc:299] Negotiation successful
0805 22:42:28.535193 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260805 22:42:28.541630 30500 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:28.542169 30500 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:28.546523 30500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543: Generated new TSK 0
I20260805 22:42:28.546705 30500 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:28.579351 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36331
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:28.669047 30510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:28.669278 30510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:28.669303 30510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:28.670464 30510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:28.670506 30510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:28.670533 30510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:28.671993 30510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:28.672204 30510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:28.672382 30510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:28.677325 30516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:28.677328 30518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:28.677333 30515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:28.677699 30510 server_base.cc:1034] running on GCE node
I20260805 22:42:28.677906 30510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:28.678120 30510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:28.679327 30510 hybrid_clock.cc:648] HybridClock initialized: now 1785969748679305 us; error 36 us; skew 500 ppm
Aug 05 22:42:28 dist-test-slave-g170 krb5kdc[30343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969748, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:28.682111 30510 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:28.682505 30510 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:28.682559 30510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:28.682744 30510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:28.683967 30510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "2e73fa410ec8469bbb97b2842bc799ad"
format_stamp: "Formatted at 2026-08-05 22:42:28 on dist-test-slave-g170"
server_key: "5ad5358b7c3e592a3c93f0b8c20f929d"
server_key_iv: "f59371341b2e8b3448d40f1d01d41d69"
server_key_version: "encryptionkey@0"
I20260805 22:42:28.684443 30510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "2e73fa410ec8469bbb97b2842bc799ad"
format_stamp: "Formatted at 2026-08-05 22:42:28 on dist-test-slave-g170"
server_key: "5ad5358b7c3e592a3c93f0b8c20f929d"
server_key_iv: "f59371341b2e8b3448d40f1d01d41d69"
server_key_version: "encryptionkey@0"
I20260805 22:42:28.686331 30510 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260805 22:42:28.687140 30525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:28.687328 30510 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:28.687428 30510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "2e73fa410ec8469bbb97b2842bc799ad"
format_stamp: "Formatted at 2026-08-05 22:42:28 on dist-test-slave-g170"
server_key: "5ad5358b7c3e592a3c93f0b8c20f929d"
server_key_iv: "f59371341b2e8b3448d40f1d01d41d69"
server_key_version: "encryptionkey@0"
I20260805 22:42:28.687518 30510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:28.700488 30510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:28.702488 30510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:28.702659 30510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:28.702912 30510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:28.703240 30510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:28.703298 30510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:28.703338 30510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:28.703357 30510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:28.709625 30510 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:37251
I20260805 22:42:28.709668 30696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:37251 every 8 connection(s)
I20260805 22:42:28.709950 30510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:28.715503 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 30510
I20260805 22:42:28.715631 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:28.715891 24341 external_mini_cluster.cc:1428] Setting key 70ff1fa15614730016b9da92e825b8b7
Aug 05 22:42:28 dist-test-slave-g170 krb5kdc[30343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969748, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:28.719485 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:28.710668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:41019 (local address 127.23.197.126:36331)
0805 22:42:28.710745 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:28.710747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:28.710966 (+ 219us) server_negotiation.cc:407] Connection header received
0805 22:42:28.711423 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:28.711424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:28.711435 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:28.711483 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:28.712410 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.712947 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.713958 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.714173 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.714245 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:28.716531 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:28.716533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:28.716533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:28.716543 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:28.717995 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:28.718471 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:28.718473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:28.718473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:28.718535 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:28.718805 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:28.718806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:28.718806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:28.718923 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:28.718999 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:28.719299 (+ 300us) server_negotiation.cc:299] Negotiation successful
0805 22:42:28.719395 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 22:42:28.719686 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:28.710647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36331 (local address 127.23.197.65:41019)
0805 22:42:28.710838 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:28.710875 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:28.711339 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:28.711542 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:28.711545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:28.711816 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:28.712326 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:28.712331 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.713027 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.713028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:28.713866 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:28.713868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.714265 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.714266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:28.714370 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:28.714844 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:28.714852 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:28.716428 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:28.718120 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:28.718128 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:28.718130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:28.718377 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:28.718613 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:28.718613 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:28.718614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:28.718730 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:28.719003 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:28.719011 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:28.719152 (+ 141us) client_negotiation.cc:769] Sending connection context
0805 22:42:28.719264 (+ 112us) client_negotiation.cc:240] Negotiation successful
0805 22:42:28.719427 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260805 22:42:28.720160 30697 heartbeater.cc:347] Connected to a master server at 127.23.197.126:36331
I20260805 22:42:28.720260 30697 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:28.720431 30697 heartbeater.cc:511] Master 127.23.197.126:36331 requested a full tablet report, sending...
I20260805 22:42:28.720991 30410 ts_manager.cc:194] Registered new tserver with Master: 2e73fa410ec8469bbb97b2842bc799ad (127.23.197.65:37251)
I20260805 22:42:28.721612 30410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:41019
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:28.745026 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36331
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:28.834169 30706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:28.834378 30706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:28.834403 30706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:28.835471 30706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:28.835512 30706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:28.835539 30706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:28.837045 30706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:28.837237 30706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:28.837419 30706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:28.842708 30712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:28.842700 30716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:28.842700 30714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:28.843089 30706 server_base.cc:1034] running on GCE node
I20260805 22:42:28.843367 30706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:28.843642 30706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:28.844820 30706 hybrid_clock.cc:648] HybridClock initialized: now 1785969748844805 us; error 35 us; skew 500 ppm
Aug 05 22:42:28 dist-test-slave-g170 krb5kdc[30343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969748, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:28.847608 30706 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:28.847973 30706 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:28.848026 30706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:28.848120 30706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:28.849341 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "ad0e2c15239c4fe08e6378373e0d814a"
format_stamp: "Formatted at 2026-08-05 22:42:28 on dist-test-slave-g170"
server_key: "246ca85287455e0629c50d20f31a1591"
server_key_iv: "a369cc85fc21a967fca98588d7519e33"
server_key_version: "encryptionkey@0"
I20260805 22:42:28.849660 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "ad0e2c15239c4fe08e6378373e0d814a"
format_stamp: "Formatted at 2026-08-05 22:42:28 on dist-test-slave-g170"
server_key: "246ca85287455e0629c50d20f31a1591"
server_key_iv: "a369cc85fc21a967fca98588d7519e33"
server_key_version: "encryptionkey@0"
I20260805 22:42:28.851058 30706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:42:28.852123 30722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:28.852298 30706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:28.852411 30706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "ad0e2c15239c4fe08e6378373e0d814a"
format_stamp: "Formatted at 2026-08-05 22:42:28 on dist-test-slave-g170"
server_key: "246ca85287455e0629c50d20f31a1591"
server_key_iv: "a369cc85fc21a967fca98588d7519e33"
server_key_version: "encryptionkey@0"
I20260805 22:42:28.852509 30706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:28.863812 30706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:28.866113 30706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:28.866308 30706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:28.866606 30706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:28.866984 30706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:28.867048 30706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:28.867090 30706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:28.867142 30706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:28.873867 30706 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:43749
I20260805 22:42:28.873919 30898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:43749 every 8 connection(s)
I20260805 22:42:28.874270 30706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:28.881568 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 30706
Aug 05 22:42:28 dist-test-slave-g170 krb5kdc[30343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969748, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:28.881716 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:28.881956 24341 external_mini_cluster.cc:1428] Setting key 0e468278ad6f742c03ef270ad9303fbb
I20260805 22:42:28.885937 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:28.875021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:56473 (local address 127.23.197.126:36331)
0805 22:42:28.875095 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:28.875097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:28.875328 (+ 231us) server_negotiation.cc:407] Connection header received
0805 22:42:28.875925 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:28.875926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:28.875940 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:28.876141 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:28.877381 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.878002 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.879271 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.879495 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.879578 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:28.882433 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:28.882435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:28.882435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:28.882444 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:28.884172 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:28.884838 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:28.884839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:28.884839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:28.884901 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:28.885218 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:28.885220 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:28.885220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:28.885344 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:28.885423 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:28.885748 (+ 325us) server_negotiation.cc:299] Negotiation successful
0805 22:42:28.885847 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 22:42:28.886171 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:28.874984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36331 (local address 127.23.197.66:56473)
0805 22:42:28.875191 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:28.875227 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:28.875740 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:28.876237 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:28.876241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:28.876563 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:28.877267 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:28.877274 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.878276 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.878278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:28.879165 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:28.879168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:28.879585 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:28.879587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:28.879718 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:28.880348 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:28.880357 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:28.882229 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:28.884277 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:28.884286 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:28.884289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:28.884680 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:28.884997 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:28.884998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:28.884998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:28.885134 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:28.885425 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:28.885432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:28.885583 (+ 151us) client_negotiation.cc:769] Sending connection context
0805 22:42:28.885710 (+ 127us) client_negotiation.cc:240] Negotiation successful
0805 22:42:28.885883 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":121,"threads_started":1}
I20260805 22:42:28.886716 30901 heartbeater.cc:347] Connected to a master server at 127.23.197.126:36331
I20260805 22:42:28.886817 30901 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:28.886992 30901 heartbeater.cc:511] Master 127.23.197.126:36331 requested a full tablet report, sending...
I20260805 22:42:28.887454 30410 ts_manager.cc:194] Registered new tserver with Master: ad0e2c15239c4fe08e6378373e0d814a (127.23.197.66:43749)
I20260805 22:42:28.887852 30410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:56473
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:28.911914 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36331
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:29.002386 30906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:29.002607 30906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:29.002632 30906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:29.003842 30906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:29.003885 30906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:29.003912 30906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:29.005537 30906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:29.005730 30906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:29.005913 30906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:29.011644 30918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:29.011730 30912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:29.011731 30915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:29.011763 30906 server_base.cc:1034] running on GCE node
I20260805 22:42:29.012244 30906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:29.012495 30906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:29.013671 30906 hybrid_clock.cc:648] HybridClock initialized: now 1785969749013656 us; error 32 us; skew 500 ppm
Aug 05 22:42:29 dist-test-slave-g170 krb5kdc[30343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969749, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:29.016395 30906 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:29.016897 30906 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:29.016979 30906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:29.017120 30906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:29.018185 30906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "bb6d1d70a1e84f60bb5480680b1baea1"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "b2ac2702922d3f939940f0513a913cce"
server_key_iv: "a68035920846b23731995ea94a024a05"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.018592 30906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "bb6d1d70a1e84f60bb5480680b1baea1"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "b2ac2702922d3f939940f0513a913cce"
server_key_iv: "a68035920846b23731995ea94a024a05"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.020078 30906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:29.020876 30926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.021080 30906 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:29.021186 30906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "bb6d1d70a1e84f60bb5480680b1baea1"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "b2ac2702922d3f939940f0513a913cce"
server_key_iv: "a68035920846b23731995ea94a024a05"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.021271 30906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:29.031991 30906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:29.034088 30906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:29.034261 30906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:29.034492 30906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:29.034832 30906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:29.034888 30906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.034922 30906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:29.034950 30906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.041288 30906 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:41377
I20260805 22:42:29.041329 31091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:41377 every 8 connection(s)
I20260805 22:42:29.041595 30906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:29.048086 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 30906
Aug 05 22:42:29 dist-test-slave-g170 krb5kdc[30343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969749, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:29.048309 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:29.048537 24341 external_mini_cluster.cc:1428] Setting key 98860d28b80715b9b36ada7b10bb16e4
I20260805 22:42:29.052009 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.042339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:35333 (local address 127.23.197.126:36331)
0805 22:42:29.042400 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.042402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.042623 (+ 221us) server_negotiation.cc:407] Connection header received
0805 22:42:29.043100 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.043101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.043110 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.043168 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.044166 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.044681 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.045872 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.046067 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.046128 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.048906 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.048908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.048908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:29.048917 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.050559 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:29.050990 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:29.050992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:29.050992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:29.051061 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:29.051339 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:29.051340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:29.051341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:29.051464 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.051542 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.051837 (+ 295us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.051918 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 22:42:29.052225 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.042329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36331 (local address 127.23.197.67:35333)
0805 22:42:29.042494 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.042537 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.043017 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.043221 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.043225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.043483 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:29.043986 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.043991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.044829 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.044831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.045789 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.045792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.046166 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.046167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.046274 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.046795 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:29.046804 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:29.048717 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:29.050668 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:29.050676 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:29.050678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:29.050908 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:29.051136 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:29.051137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:29.051138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:29.051261 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:29.051532 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:29.051540 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:29.051692 (+ 152us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.051800 (+ 108us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.051963 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260805 22:42:29.052642 31092 heartbeater.cc:347] Connected to a master server at 127.23.197.126:36331
I20260805 22:42:29.052735 31092 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:29.052953 31092 heartbeater.cc:511] Master 127.23.197.126:36331 requested a full tablet report, sending...
I20260805 22:42:29.053335 30410 ts_manager.cc:194] Registered new tserver with Master: bb6d1d70a1e84f60bb5480680b1baea1 (127.23.197.67:41377)
I20260805 22:42:29.053725 30410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:35333
I20260805 22:42:29.060340 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:42:29.067845 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.061852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49924 (local address 127.23.197.126:36331)
0805 22:42:29.061897 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.061900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.061998 (+ 98us) server_negotiation.cc:407] Connection header received
0805 22:42:29.062053 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.062054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.062063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.062113 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.062696 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.063165 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.063856 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.064042 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.064099 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.065321 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.065322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.065322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:29.065328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.066772 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:29.067081 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:29.067082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:29.067082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:29.067133 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:29.067295 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:29.067295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:29.067295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:29.067431 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.067505 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.067598 (+ 93us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.067697 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:42:29.069635 30410 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:29.071280 30410 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:29.077945 31107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.073749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:43749 (local address 127.0.0.1:57692)
0805 22:42:29.074141 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.074182 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.074270 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.074532 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.074535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.074538 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:29.074613 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.074615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.075654 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.075655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.076707 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.076710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.077654 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.077655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.077749 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.077766 (+ 17us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.077814 (+ 48us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.077865 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":88,"threads_started":1}
I20260805 22:42:29.078085 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.073827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:41377 (local address 127.0.0.1:51826)
0805 22:42:29.074140 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.074183 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.074253 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.074737 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.074738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.074741 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:29.074787 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.074788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.076244 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.076245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.077152 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.077154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.077871 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.077872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.077925 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.077928 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.077975 (+ 47us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.078015 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":95,"threads_started":1}
I20260805 22:42:29.078146 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.073981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.23.197.67:41377)
0805 22:42:29.074434 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.074440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.074451 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:29.074491 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.074493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.074510 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.074588 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:29.074989 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.076134 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.077239 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.077735 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.077871 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.077940 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.077996 (+ 56us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.078041 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":88,"threads_started":1}
I20260805 22:42:29.078428 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.074004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57692 (local address 127.23.197.66:43749)
0805 22:42:29.074330 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.074335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.074345 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:29.074409 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.074411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.074428 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.074526 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:29.074706 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.075559 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.076808 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.077551 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.078162 (+ 611us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.078238 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.078270 (+ 32us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.078337 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":90,"threads_started":1}
I20260805 22:42:29.078879 30992 tablet_service.cc:1467] Processing CreateTablet for tablet ff19823ea0494a67864bf3d2725a7281 (DEFAULT_TABLE table=test-table [id=9749d375c1bf420394849de80fb91a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:29.079082 30795 tablet_service.cc:1467] Processing CreateTablet for tablet ff19823ea0494a67864bf3d2725a7281 (DEFAULT_TABLE table=test-table [id=9749d375c1bf420394849de80fb91a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:29.079283 30992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff19823ea0494a67864bf3d2725a7281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:29.079319 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.073995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:37251 (local address 127.0.0.1:39288)
0805 22:42:29.074637 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.074647 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.074690 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.075255 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.075256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.075259 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:29.075298 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.075300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.076516 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.076517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.077724 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.077726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.079048 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.079049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.079111 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.079116 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.079203 (+ 87us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.079243 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":85,"threads_started":1}
I20260805 22:42:29.079339 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.074315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39288 (local address 127.23.197.65:37251)
0805 22:42:29.075081 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.075085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.075095 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:29.075133 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.075135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.075150 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.075234 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:29.075376 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.076380 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.077824 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.078952 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.079045 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.079098 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.079222 (+ 124us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.079263 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":95,"threads_started":1}
I20260805 22:42:29.079401 30795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff19823ea0494a67864bf3d2725a7281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:29.079924 30597 tablet_service.cc:1467] Processing CreateTablet for tablet ff19823ea0494a67864bf3d2725a7281 (DEFAULT_TABLE table=test-table [id=9749d375c1bf420394849de80fb91a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:29.080163 30597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff19823ea0494a67864bf3d2725a7281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:29.082846 31113 tablet_bootstrap.cc:492] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad: Bootstrap starting.
I20260805 22:42:29.083467 31113 tablet_bootstrap.cc:654] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:29.084138 31113 log.cc:824] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:29.084487 31114 tablet_bootstrap.cc:492] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: Bootstrap starting.
I20260805 22:42:29.085052 31113 tablet_bootstrap.cc:492] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad: No bootstrap required, opened a new log
I20260805 22:42:29.085139 31113 ts_tablet_manager.cc:1397] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:29.085162 31114 tablet_bootstrap.cc:654] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:29.085662 31114 log.cc:824] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:29.086056 31116 tablet_bootstrap.cc:492] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: Bootstrap starting.
I20260805 22:42:29.086616 31116 tablet_bootstrap.cc:654] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:29.086910 31113 raft_consensus.cc:348] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } }
I20260805 22:42:29.087138 31116 log.cc:824] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:29.087086 31113 raft_consensus.cc:374] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:29.087271 31113 raft_consensus.cc:729] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e73fa410ec8469bbb97b2842bc799ad, State: Initialized, Role: FOLLOWER
I20260805 22:42:29.087423 31113 consensus_queue.cc:260] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } }
I20260805 22:42:29.087625 31114 tablet_bootstrap.cc:492] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: No bootstrap required, opened a new log
I20260805 22:42:29.087703 31114 ts_tablet_manager.cc:1397] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260805 22:42:29.087855 31113 ts_tablet_manager.cc:1428] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:42:29.087860 30697 heartbeater.cc:503] Master 127.23.197.126:36331 was elected leader, sending a full tablet report...
I20260805 22:42:29.089324 31114 raft_consensus.cc:348] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } }
I20260805 22:42:29.089485 31114 raft_consensus.cc:374] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:29.089533 31114 raft_consensus.cc:729] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb6d1d70a1e84f60bb5480680b1baea1, State: Initialized, Role: FOLLOWER
I20260805 22:42:29.089664 31114 consensus_queue.cc:260] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } }
I20260805 22:42:29.089968 31114 ts_tablet_manager.cc:1428] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:29.090025 31092 heartbeater.cc:503] Master 127.23.197.126:36331 was elected leader, sending a full tablet report...
I20260805 22:42:29.091207 31116 tablet_bootstrap.cc:492] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: No bootstrap required, opened a new log
I20260805 22:42:29.091312 31116 ts_tablet_manager.cc:1397] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260805 22:42:29.093201 31116 raft_consensus.cc:348] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } }
I20260805 22:42:29.093348 31116 raft_consensus.cc:374] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:29.093374 31116 raft_consensus.cc:729] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad0e2c15239c4fe08e6378373e0d814a, State: Initialized, Role: FOLLOWER
I20260805 22:42:29.093510 31116 consensus_queue.cc:260] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } }
I20260805 22:42:29.093873 31116 ts_tablet_manager.cc:1428] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260805 22:42:29.093883 30901 heartbeater.cc:503] Master 127.23.197.126:36331 was elected leader, sending a full tablet report...
W20260805 22:42:29.125042 30902 tablet.cc:2369] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:42:29.210654 30698 tablet.cc:2369] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:42:29.254272 31125 raft_consensus.cc:482] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:29.254412 31125 raft_consensus.cc:504] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } }
I20260805 22:42:29.254796 31125 leader_election.cc:288] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e73fa410ec8469bbb97b2842bc799ad (127.23.197.65:37251), bb6d1d70a1e84f60bb5480680b1baea1 (127.23.197.67:41377)
I20260805 22:42:29.259320 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.255225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:48961 (local address 127.23.197.65:37251)
0805 22:42:29.255279 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.255281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.255296 (+ 15us) server_negotiation.cc:407] Connection header received
0805 22:42:29.255331 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.255332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.255343 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.255466 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:29.255750 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.256588 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.258226 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.258815 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.258902 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.258914 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.259192 (+ 278us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.259243 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260805 22:42:29.259320 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.255240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:47919 (local address 127.23.197.67:41377)
0805 22:42:29.255288 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.255291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.255417 (+ 126us) server_negotiation.cc:407] Connection header received
0805 22:42:29.255450 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.255451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.255463 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.255528 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:29.255750 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.256589 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.258226 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.258862 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.258929 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.258937 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.259192 (+ 255us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.259243 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:42:29.259294 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.255076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:37251 (local address 127.23.197.66:48961)
0805 22:42:29.255172 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.255186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.255242 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.255496 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.255497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.255520 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:29.255655 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.255657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.256692 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.256693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.258110 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.258113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.258944 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.258945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.259055 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.259057 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.259134 (+ 77us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.259185 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":53}
I20260805 22:42:29.259300 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.255097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:41377 (local address 127.23.197.66:47919)
0805 22:42:29.255322 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.255332 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.255393 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.255543 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.255544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.255545 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:29.255610 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.255612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.256702 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.256703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.258118 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:29.258119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.258945 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.258946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:29.259052 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.259057 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.259148 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.259188 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":71,"thread_start_us":138,"threads_started":1}
I20260805 22:42:29.259876 30625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff19823ea0494a67864bf3d2725a7281" candidate_uuid: "ad0e2c15239c4fe08e6378373e0d814a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e73fa410ec8469bbb97b2842bc799ad" is_pre_election: true
I20260805 22:42:29.259865 31019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff19823ea0494a67864bf3d2725a7281" candidate_uuid: "ad0e2c15239c4fe08e6378373e0d814a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" is_pre_election: true
I20260805 22:42:29.260046 30625 raft_consensus.cc:2454] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad0e2c15239c4fe08e6378373e0d814a in term 0.
I20260805 22:42:29.260056 31019 raft_consensus.cc:2454] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad0e2c15239c4fe08e6378373e0d814a in term 0.
I20260805 22:42:29.260324 30725 leader_election.cc:302] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2e73fa410ec8469bbb97b2842bc799ad, ad0e2c15239c4fe08e6378373e0d814a, bb6d1d70a1e84f60bb5480680b1baea1; no voters:
I20260805 22:42:29.260730 31125 raft_consensus.cc:2781] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:29.260829 31125 raft_consensus.cc:482] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:29.260866 31125 raft_consensus.cc:3037] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:29.261641 31125 raft_consensus.cc:504] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } }
I20260805 22:42:29.261818 31125 leader_election.cc:288] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [CANDIDATE]: Term 1 election: Requested vote from peers 2e73fa410ec8469bbb97b2842bc799ad (127.23.197.65:37251), bb6d1d70a1e84f60bb5480680b1baea1 (127.23.197.67:41377)
I20260805 22:42:29.261986 31019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff19823ea0494a67864bf3d2725a7281" candidate_uuid: "ad0e2c15239c4fe08e6378373e0d814a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb6d1d70a1e84f60bb5480680b1baea1"
I20260805 22:42:29.262063 30625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff19823ea0494a67864bf3d2725a7281" candidate_uuid: "ad0e2c15239c4fe08e6378373e0d814a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e73fa410ec8469bbb97b2842bc799ad"
I20260805 22:42:29.262079 31019 raft_consensus.cc:3037] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:29.262142 30625 raft_consensus.cc:3037] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:29.263046 31019 raft_consensus.cc:2454] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad0e2c15239c4fe08e6378373e0d814a in term 1.
I20260805 22:42:29.263046 30625 raft_consensus.cc:2454] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad0e2c15239c4fe08e6378373e0d814a in term 1.
I20260805 22:42:29.263243 30730 leader_election.cc:302] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad0e2c15239c4fe08e6378373e0d814a, bb6d1d70a1e84f60bb5480680b1baea1; no voters:
I20260805 22:42:29.263353 31125 raft_consensus.cc:2781] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:29.263535 31125 raft_consensus.cc:686] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 1 LEADER]: Becoming Leader. State: Replica: ad0e2c15239c4fe08e6378373e0d814a, State: Running, Role: LEADER
I20260805 22:42:29.263670 31125 consensus_queue.cc:237] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } }
I20260805 22:42:29.264616 30410 catalog_manager.cc:5495] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a reported cstate change: term changed from 0 to 1, leader changed from <none> to ad0e2c15239c4fe08e6378373e0d814a (127.23.197.66). New cstate: current_term: 1 leader_uuid: "ad0e2c15239c4fe08e6378373e0d814a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad0e2c15239c4fe08e6378373e0d814a" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 } health_report { overall_health: UNKNOWN } } }
W20260805 22:42:29.292150 31093 tablet.cc:2369] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:42:29.307929 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.305322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.23.197.66:43749)
0805 22:42:29.305414 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.305422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.305433 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:29.305473 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.305473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.305494 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.305566 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:29.305824 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.306310 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.307095 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:29.307292 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:29.307354 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:29.307519 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:29.307625 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:29.307690 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.307826 (+ 136us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.307844 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 22:42:29.310061 30625 raft_consensus.cc:1261] T ff19823ea0494a67864bf3d2725a7281 P 2e73fa410ec8469bbb97b2842bc799ad [term 1 FOLLOWER]: Refusing update from remote peer ad0e2c15239c4fe08e6378373e0d814a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:29.310074 31019 raft_consensus.cc:1261] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [term 1 FOLLOWER]: Refusing update from remote peer ad0e2c15239c4fe08e6378373e0d814a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:29.310525 31125 consensus_queue.cc:1035] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb6d1d70a1e84f60bb5480680b1baea1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:29.310727 31128 consensus_queue.cc:1035] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e73fa410ec8469bbb97b2842bc799ad" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 37251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:29.313573 31131 mvcc.cc:205] Tried to move back new op lower bound from 7315332093170688000 to 7315332092984643584. Current Snapshot: MvccSnapshot[applied={T|T < 7315332093170688000}]
I20260805 22:42:29.314324 31137 mvcc.cc:205] Tried to move back new op lower bound from 7315332093170688000 to 7315332092984643584. Current Snapshot: MvccSnapshot[applied={T|T < 7315332093170688000}]
I20260805 22:42:29.315737 30409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:29.315834 30409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:29.316618 30409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01b4a79c5ce943c98b8a10e463dae543: Sending DeleteTablet for 3 replicas of tablet ff19823ea0494a67864bf3d2725a7281
I20260805 22:42:29.316982 30795 tablet_service.cc:1514] Processing DeleteTablet for tablet ff19823ea0494a67864bf3d2725a7281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:29 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:57692
I20260805 22:42:29.317019 30597 tablet_service.cc:1514] Processing DeleteTablet for tablet ff19823ea0494a67864bf3d2725a7281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:29 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:39288
I20260805 22:42:29.317234 31149 tablet_replica.cc:330] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: stopping tablet replica
I20260805 22:42:29.317231 30992 tablet_service.cc:1514] Processing DeleteTablet for tablet ff19823ea0494a67864bf3d2725a7281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:29 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:51826
I20260805 22:42:29.317340 31149 raft_consensus.cc:2229] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:29.317433 31149 raft_consensus.cc:2258] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:29.317471 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 30510
I20260805 22:42:29.323675 31149 ts_tablet_manager.cc:1905] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:29.325825 31151 tablet_replica.cc:330] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: stopping tablet replica
I20260805 22:42:29.325974 31151 raft_consensus.cc:2229] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:29.326092 31151 raft_consensus.cc:2258] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:29.326509 31149 ts_tablet_manager.cc:1918] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:29.326612 31149 log.cc:1196] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/ff19823ea0494a67864bf3d2725a7281
I20260805 22:42:29.327991 31151 ts_tablet_manager.cc:1905] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:29.328006 31149 ts_tablet_manager.cc:1939] T ff19823ea0494a67864bf3d2725a7281 P ad0e2c15239c4fe08e6378373e0d814a: Deleting consensus metadata
I20260805 22:42:29.328491 30390 catalog_manager.cc:4841] TS ad0e2c15239c4fe08e6378373e0d814a (127.23.197.66:43749): tablet ff19823ea0494a67864bf3d2725a7281 (table test-table [id=9749d375c1bf420394849de80fb91a66]) successfully deleted
I20260805 22:42:29.329794 31151 ts_tablet_manager.cc:1918] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:29.329870 31151 log.cc:1196] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/ff19823ea0494a67864bf3d2725a7281
I20260805 22:42:29.330157 31151 ts_tablet_manager.cc:1939] T ff19823ea0494a67864bf3d2725a7281 P bb6d1d70a1e84f60bb5480680b1baea1: Deleting consensus metadata
I20260805 22:42:29.330648 30392 catalog_manager.cc:4841] TS bb6d1d70a1e84f60bb5480680b1baea1 (127.23.197.67:41377): tablet ff19823ea0494a67864bf3d2725a7281 (table test-table [id=9749d375c1bf420394849de80fb91a66]) successfully deleted
W20260805 22:42:29.336443 30390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:37251 (error 108)
I20260805 22:42:29.337143 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.336981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:37251 (local address 127.0.0.1:39292)
0805 22:42:29.337050 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.337096 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260805 22:42:29.337214 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 30706
W20260805 22:42:29.342263 30390 catalog_manager.cc:4584] TS 2e73fa410ec8469bbb97b2842bc799ad (127.23.197.65:37251): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff19823ea0494a67864bf3d2725a7281: Network error: Client connection negotiation failed: client connection to 127.23.197.65:37251: connect: Connection refused (error 111)
I20260805 22:42:29.346714 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 30906
I20260805 22:42:29.362368 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 30360
2026-08-05T22:42:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2350 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-05T22:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:29Z Disabled control of system clock
I20260805 22:42:29.395960 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:34675
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:34675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260805 22:42:29.485008 31158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:29.485241 31158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:29.485275 31158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:29.486498 31158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:29.486541 31158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:29.486555 31158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:29.486604 31158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:29.486631 31158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:29.488149 31158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:34675
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:34675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:29.488379 31158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:29.488657 31158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:29.494158 31166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:29.494243 31165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:29.494316 31170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:29.494171 31158 server_base.cc:1034] running on GCE node
I20260805 22:42:29.494599 31158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:29.495045 31158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:29.496214 31158 hybrid_clock.cc:648] HybridClock initialized: now 1785969749496203 us; error 31 us; skew 500 ppm
I20260805 22:42:29.497669 31158 webserver.cc:460] Webserver started at http://127.23.197.126:43043/ using document root <none> and password file <none>
I20260805 22:42:29.497979 31158 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:29.498029 31158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:29.498183 31158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:29.499320 31158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "686a68e5561042b28ce7b7f7843f9208"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "432d071e2423845b51810cfe63931cf5"
server_key_iv: "2822996966ec2cd7b28395330fa91978"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.499708 31158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "686a68e5561042b28ce7b7f7843f9208"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "432d071e2423845b51810cfe63931cf5"
server_key_iv: "2822996966ec2cd7b28395330fa91978"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.501129 31158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:29.501952 31188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.502177 31158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:29.502290 31158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "686a68e5561042b28ce7b7f7843f9208"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "432d071e2423845b51810cfe63931cf5"
server_key_iv: "2822996966ec2cd7b28395330fa91978"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.502367 31158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:29.505946 31158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:29.506227 31158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:29.506376 31158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:29.510675 31158 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:34675
I20260805 22:42:29.510713 31266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:34675 every 8 connection(s)
I20260805 22:42:29.511094 31158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:29.511770 31267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:29.514564 31267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208: Bootstrap starting.
I20260805 22:42:29.515100 31267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:29.515347 31267 log.cc:824] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:29.516027 31267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208: No bootstrap required, opened a new log
I20260805 22:42:29.517699 31267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686a68e5561042b28ce7b7f7843f9208" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34675 } }
I20260805 22:42:29.517848 31267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:29.517871 31267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 686a68e5561042b28ce7b7f7843f9208, State: Initialized, Role: FOLLOWER
I20260805 22:42:29.518111 31267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686a68e5561042b28ce7b7f7843f9208" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34675 } }
I20260805 22:42:29.518239 31267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:29.518266 31267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:29.518301 31267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:29.518870 31267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686a68e5561042b28ce7b7f7843f9208" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34675 } }
I20260805 22:42:29.518961 31267 leader_election.cc:302] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 686a68e5561042b28ce7b7f7843f9208; no voters:
I20260805 22:42:29.519093 31267 leader_election.cc:288] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:29.519362 31273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:29.519574 31267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:29.519583 31273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [term 1 LEADER]: Becoming Leader. State: Replica: 686a68e5561042b28ce7b7f7843f9208, State: Running, Role: LEADER
I20260805 22:42:29.519817 31273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686a68e5561042b28ce7b7f7843f9208" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34675 } }
I20260805 22:42:29.520228 31275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 686a68e5561042b28ce7b7f7843f9208. Latest consensus state: current_term: 1 leader_uuid: "686a68e5561042b28ce7b7f7843f9208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686a68e5561042b28ce7b7f7843f9208" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34675 } } }
I20260805 22:42:29.520318 31275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:29.520566 31274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "686a68e5561042b28ce7b7f7843f9208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686a68e5561042b28ce7b7f7843f9208" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34675 } } }
I20260805 22:42:29.520680 31274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:29.521101 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 31158
I20260805 22:42:29.521238 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:29.521442 24341 external_mini_cluster.cc:1428] Setting key 69072d340e09ae717bab26d449b936df
W20260805 22:42:29.522045 31301 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:42:29.522161 31301 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:42:29.522287 31289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:29.522476 31289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:29.524114 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.522157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45638 (local address 127.23.197.126:34675)
0805 22:42:29.522402 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.522412 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.522446 (+ 34us) server_negotiation.cc:407] Connection header received
0805 22:42:29.522566 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.522574 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.522647 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.522810 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.523531 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.523534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.523535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:29.523539 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.523614 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:29.523631 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.523699 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.523844 (+ 145us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.523908 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":78,"threads_started":1}
I20260805 22:42:29.524199 31289 catalog_manager.cc:1347] Generated new cluster ID: 1b87316b8ed2453dbdca66c5eed9475e
I20260805 22:42:29.524282 31289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:29.549536 31289 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:29.549825 31289 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:29.557021 31289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208: Generated new TSK 0
I20260805 22:42:29.557271 31289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:42:29.565500 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:34675
--builtin_ntp_servers=127.23.197.84:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260805 22:42:29.651533 31308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:29.651750 31308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:29.651774 31308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:29.652995 31308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:29.653034 31308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:29.653059 31308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:29.654414 31308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:34675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:29.654657 31308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:29.654991 31308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:29.661428 31320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:29.661455 31317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:29.661419 31315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:29.661703 31308 server_base.cc:1034] running on GCE node
I20260805 22:42:29.661995 31308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:29.662282 31308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:29.663458 31308 hybrid_clock.cc:648] HybridClock initialized: now 1785969749663429 us; error 53 us; skew 500 ppm
I20260805 22:42:29.665233 31308 webserver.cc:460] Webserver started at http://127.23.197.65:43337/ using document root <none> and password file <none>
I20260805 22:42:29.665510 31308 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:29.665573 31308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:29.665727 31308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:29.667234 31308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "a419491a3ed0439187220e228c592fda"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "3b8d54c8e419eae6d39b1653db2a8c42"
server_key_iv: "26ca78788bc16de851f8f679a4f7e50f"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.667727 31308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "a419491a3ed0439187220e228c592fda"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "3b8d54c8e419eae6d39b1653db2a8c42"
server_key_iv: "26ca78788bc16de851f8f679a4f7e50f"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.669549 31308 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260805 22:42:29.670432 31329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.670753 31308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:29.670888 31308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "a419491a3ed0439187220e228c592fda"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "3b8d54c8e419eae6d39b1653db2a8c42"
server_key_iv: "26ca78788bc16de851f8f679a4f7e50f"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.670982 31308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:29.674533 31308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:29.674825 31308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:29.674978 31308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:29.675202 31308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:29.675521 31308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:29.675578 31308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.675617 31308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:29.675644 31308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.681852 31308 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:45237
I20260805 22:42:29.681895 31488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:45237 every 8 connection(s)
I20260805 22:42:29.682253 31308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:29.685616 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.683074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:40951 (local address 127.23.197.126:34675)
0805 22:42:29.683153 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.683155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.683365 (+ 210us) server_negotiation.cc:407] Connection header received
0805 22:42:29.683912 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.683913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.683919 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.683991 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.685357 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.685358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.685358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:29.685363 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.685380 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:29.685384 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.685458 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.685547 (+ 89us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.685561 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 22:42:29.685829 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.683027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:34675 (local address 127.23.197.65:40951)
0805 22:42:29.683224 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.683265 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.683732 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.684100 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.684104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.684113 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:29.685197 (+ 1084us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:42:29.685198 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:29.685229 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:42:29.685259 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:29.685451 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:29.685457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:29.685460 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.685535 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.685587 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260805 22:42:29.686322 31489 heartbeater.cc:347] Connected to a master server at 127.23.197.126:34675
I20260805 22:42:29.686404 31489 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:29.686587 31489 heartbeater.cc:511] Master 127.23.197.126:34675 requested a full tablet report, sending...
I20260805 22:42:29.687132 31218 ts_manager.cc:194] Registered new tserver with Master: a419491a3ed0439187220e228c592fda (127.23.197.65:45237)
I20260805 22:42:29.691793 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 31308
I20260805 22:42:29.691892 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:29.692034 24341 external_mini_cluster.cc:1428] Setting key 11a77ee2ce33c0ccf9b13c79f100a668
I20260805 22:42:29.693173 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:34675
--builtin_ntp_servers=127.23.197.84:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260805 22:42:29.778409 31495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:29.778774 31495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:29.778868 31495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:29.780126 31495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:29.780215 31495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:29.780267 31495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:29.782013 31495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:34675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:29.782362 31495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:29.782706 31495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:29.788288 31508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:29.788292 31507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:29.788439 31495 server_base.cc:1034] running on GCE node
W20260805 22:42:29.788350 31510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:29.788888 31495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:29.789172 31495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:29.790355 31495 hybrid_clock.cc:648] HybridClock initialized: now 1785969749790346 us; error 41 us; skew 500 ppm
I20260805 22:42:29.791657 31495 webserver.cc:460] Webserver started at http://127.23.197.66:35817/ using document root <none> and password file <none>
I20260805 22:42:29.791980 31495 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:29.792068 31495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:29.792240 31495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:29.793473 31495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "0be3c756baef4228b76d483e91715083"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "9649013c45690fc77c25caabab77a6b1"
server_key_iv: "fe0544a16b1d5ee812adf01ce305b849"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.793869 31495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "0be3c756baef4228b76d483e91715083"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "9649013c45690fc77c25caabab77a6b1"
server_key_iv: "fe0544a16b1d5ee812adf01ce305b849"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.795333 31495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:42:29.796528 31516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.796888 31495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:29.797004 31495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "0be3c756baef4228b76d483e91715083"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "9649013c45690fc77c25caabab77a6b1"
server_key_iv: "fe0544a16b1d5ee812adf01ce305b849"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.797096 31495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:29.800649 31495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:29.800956 31495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:29.801106 31495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:29.801334 31495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:29.801671 31495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:29.801728 31495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.801760 31495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:29.801789 31495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.807931 31495 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:43343
I20260805 22:42:29.808015 31699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:43343 every 8 connection(s)
I20260805 22:42:29.808367 31495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:29.809010 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 31495
I20260805 22:42:29.809134 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:29.809355 24341 external_mini_cluster.cc:1428] Setting key bc632b166f4325ed560fe081815d8c9b
I20260805 22:42:29.810662 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:34675
--builtin_ntp_servers=127.23.197.84:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260805 22:42:29.812148 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.809156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:56393 (local address 127.23.197.126:34675)
0805 22:42:29.809566 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.809568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.809580 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:42:29.810194 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.810195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.810202 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.810250 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.811843 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.811844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.811845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:29.811849 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.811866 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:29.811870 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.811932 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.812077 (+ 145us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.812093 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380}
I20260805 22:42:29.812373 31702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.809166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:34675 (local address 127.23.197.66:56393)
0805 22:42:29.809401 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.809454 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.810064 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.810325 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.810329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.810340 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:29.811644 (+ 1304us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:42:29.811645 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:29.811687 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:42:29.811724 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:29.811951 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:29.811957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:29.811960 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.812041 (+ 81us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.812123 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260805 22:42:29.812963 31700 heartbeater.cc:347] Connected to a master server at 127.23.197.126:34675
I20260805 22:42:29.813079 31700 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:29.813248 31700 heartbeater.cc:511] Master 127.23.197.126:34675 requested a full tablet report, sending...
I20260805 22:42:29.813640 31218 ts_manager.cc:194] Registered new tserver with Master: 0be3c756baef4228b76d483e91715083 (127.23.197.66:43343)
W20260805 22:42:29.897668 31703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:29.897884 31703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:29.897908 31703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:29.899179 31703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:29.899224 31703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:29.899251 31703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:29.900893 31703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:34675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:29.901166 31703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:29.901496 31703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:29.906960 31713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:29.907115 31703 server_base.cc:1034] running on GCE node
W20260805 22:42:29.906986 31710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:29.907083 31709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:29.907475 31703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:29.907833 31703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:29.908991 31703 hybrid_clock.cc:648] HybridClock initialized: now 1785969749908983 us; error 36 us; skew 500 ppm
I20260805 22:42:29.910276 31703 webserver.cc:460] Webserver started at http://127.23.197.67:43547/ using document root <none> and password file <none>
I20260805 22:42:29.910522 31703 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:29.910567 31703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:29.910665 31703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:29.911741 31703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "bef53706199546a690a9e7cdfe488d38"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "2674638656b30fd9ce23cfa0286ba3b2"
server_key_iv: "0ca7d62c68a4db809d5f95ec16440af2"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.912050 31703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "bef53706199546a690a9e7cdfe488d38"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "2674638656b30fd9ce23cfa0286ba3b2"
server_key_iv: "0ca7d62c68a4db809d5f95ec16440af2"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.913481 31703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:29.914270 31727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.914479 31703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:29.914589 31703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "bef53706199546a690a9e7cdfe488d38"
format_stamp: "Formatted at 2026-08-05 22:42:29 on dist-test-slave-g170"
server_key: "2674638656b30fd9ce23cfa0286ba3b2"
server_key_iv: "0ca7d62c68a4db809d5f95ec16440af2"
server_key_version: "encryptionkey@0"
I20260805 22:42:29.914665 31703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:29.918278 31703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:29.918576 31703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:29.918728 31703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:29.918944 31703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:29.919315 31703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:29.919363 31703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.919402 31703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:29.919430 31703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:29.925896 31703 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:36645
I20260805 22:42:29.925940 31894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:36645 every 8 connection(s)
I20260805 22:42:29.926293 31703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:29.927187 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 31703
I20260805 22:42:29.927297 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:29.927438 24341 external_mini_cluster.cc:1428] Setting key 0c5e49ac7c9925f3e409e58a02418998
I20260805 22:42:29.929993 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.927063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:33821 (local address 127.23.197.126:34675)
0805 22:42:29.927123 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.927126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.927826 (+ 700us) server_negotiation.cc:407] Connection header received
0805 22:42:29.928450 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.928451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.928456 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.928486 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.929755 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.929756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.929756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:29.929761 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.929776 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:29.929780 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.929851 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.929927 (+ 76us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.929939 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260805 22:42:29.930212 31900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.927028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:34675 (local address 127.23.197.67:33821)
0805 22:42:29.927676 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.927728 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.928344 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.928552 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.928555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.928564 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:29.929517 (+ 953us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:42:29.929517 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:29.929547 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:42:29.929576 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:29.929863 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:29.929869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:29.929872 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.929936 (+ 64us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.929988 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":88,"threads_started":1}
I20260805 22:42:29.930879 31895 heartbeater.cc:347] Connected to a master server at 127.23.197.126:34675
I20260805 22:42:29.930990 31895 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:29.931172 31895 heartbeater.cc:511] Master 127.23.197.126:34675 requested a full tablet report, sending...
I20260805 22:42:29.931584 31218 ts_manager.cc:194] Registered new tserver with Master: bef53706199546a690a9e7cdfe488d38 (127.23.197.67:36645)
I20260805 22:42:29.939574 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:42:29.942418 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.941126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45646 (local address 127.23.197.126:34675)
0805 22:42:29.941257 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.941260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.941325 (+ 65us) server_negotiation.cc:407] Connection header received
0805 22:42:29.941399 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.941400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.941408 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.941473 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.942141 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.942143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.942143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:29.942148 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.942164 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:29.942169 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.942252 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.942321 (+ 69us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.942339 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260805 22:42:29.944519 31218 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:29.946264 31218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:29.951009 31914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.948970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:36645 (local address 127.0.0.1:44112)
0805 22:42:29.949355 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.949363 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.949414 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.949988 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.949994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.950001 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:29.950526 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:42:29.950527 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:29.950546 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:42:29.950569 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:29.950838 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:29.950846 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:29.950849 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.950915 (+ 66us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.950930 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":112,"threads_started":1}
I20260805 22:42:29.951097 31916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.949267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.23.197.67:36645)
0805 22:42:29.949500 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.949504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.949515 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:29.949544 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.949546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.949556 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.949629 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.950640 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.950648 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.950648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:29.950652 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.950692 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:29.950710 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.950811 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.950957 (+ 146us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.950979 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":107,"threads_started":1}
I20260805 22:42:29.951232 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.949306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.23.197.65:45237)
0805 22:42:29.949528 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.949533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.949546 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:42:29.949578 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.949580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.949591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.949688 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.950818 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.950831 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.950831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:29.950837 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.950878 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:29.950906 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.950973 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.951100 (+ 127us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.951122 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":116,"threads_started":1}
I20260805 22:42:29.951167 31910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.948965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:45237 (local address 127.0.0.1:51014)
0805 22:42:29.949157 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.949199 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.949266 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.949993 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.949994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.950003 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:29.950688 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:42:29.950689 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:29.950697 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:42:29.950712 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:29.951003 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:29.951004 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:29.951005 (+ 1us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.951065 (+ 60us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.951089 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":64,"threads_started":1}
I20260805 22:42:29.951711 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.948970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:43343 (local address 127.0.0.1:44302)
0805 22:42:29.949157 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:29.949199 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:29.949259 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:29.950462 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:29.950463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:29.950466 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:29.950864 (+ 398us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:42:29.950865 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:29.950871 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:42:29.950882 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:29.951536 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:29.951537 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:29.951538 (+ 1us) client_negotiation.cc:769] Sending connection context
0805 22:42:29.951618 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 22:42:29.951633 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":117,"threads_started":1}
I20260805 22:42:29.952152 31806 tablet_service.cc:1467] Processing CreateTablet for tablet d0088c8f9763492da1e926f19223f652 (DEFAULT_TABLE table=test-table [id=41a6f705c50b41ecbd9f71a709e0f4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:29.952152 31393 tablet_service.cc:1467] Processing CreateTablet for tablet d0088c8f9763492da1e926f19223f652 (DEFAULT_TABLE table=test-table [id=41a6f705c50b41ecbd9f71a709e0f4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:29.952454 31393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0088c8f9763492da1e926f19223f652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:29.952458 31806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0088c8f9763492da1e926f19223f652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:29.953042 31919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:29.949557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.23.197.66:43343)
0805 22:42:29.950286 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:29.950291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:29.950302 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:29.950332 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:29.950334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:29.950345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:29.950447 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:29.951407 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:29.951411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:29.951412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:29.951416 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:29.951450 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:29.951465 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:29.951538 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:29.952928 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0805 22:42:29.952952 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":93,"threads_started":1}
I20260805 22:42:29.953694 31592 tablet_service.cc:1467] Processing CreateTablet for tablet d0088c8f9763492da1e926f19223f652 (DEFAULT_TABLE table=test-table [id=41a6f705c50b41ecbd9f71a709e0f4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:29.954046 31592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0088c8f9763492da1e926f19223f652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:29.955646 31925 tablet_bootstrap.cc:492] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda: Bootstrap starting.
I20260805 22:42:29.956278 31925 tablet_bootstrap.cc:654] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:29.956617 31925 log.cc:824] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:29.957320 31927 tablet_bootstrap.cc:492] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: Bootstrap starting.
I20260805 22:42:29.957377 31920 tablet_bootstrap.cc:492] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: Bootstrap starting.
I20260805 22:42:29.957530 31925 tablet_bootstrap.cc:492] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda: No bootstrap required, opened a new log
I20260805 22:42:29.957614 31925 ts_tablet_manager.cc:1397] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260805 22:42:29.958045 31927 tablet_bootstrap.cc:654] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:29.958112 31920 tablet_bootstrap.cc:654] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:29.958436 31920 log.cc:824] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:29.958491 31927 log.cc:824] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:29.959149 31920 tablet_bootstrap.cc:492] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: No bootstrap required, opened a new log
I20260805 22:42:29.959205 31920 ts_tablet_manager.cc:1397] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:42:29.960031 31925 raft_consensus.cc:348] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } }
I20260805 22:42:29.960191 31925 raft_consensus.cc:374] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:29.960242 31925 raft_consensus.cc:729] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a419491a3ed0439187220e228c592fda, State: Initialized, Role: FOLLOWER
I20260805 22:42:29.960391 31925 consensus_queue.cc:260] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } }
I20260805 22:42:29.960700 31925 ts_tablet_manager.cc:1428] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:42:29.960711 31489 heartbeater.cc:503] Master 127.23.197.126:34675 was elected leader, sending a full tablet report...
I20260805 22:42:29.961390 31920 raft_consensus.cc:348] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } }
I20260805 22:42:29.961540 31920 raft_consensus.cc:374] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:29.961576 31920 raft_consensus.cc:729] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef53706199546a690a9e7cdfe488d38, State: Initialized, Role: FOLLOWER
I20260805 22:42:29.961685 31920 consensus_queue.cc:260] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } }
I20260805 22:42:29.961949 31920 ts_tablet_manager.cc:1428] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:42:29.961980 31895 heartbeater.cc:503] Master 127.23.197.126:34675 was elected leader, sending a full tablet report...
I20260805 22:42:29.962618 31927 tablet_bootstrap.cc:492] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: No bootstrap required, opened a new log
I20260805 22:42:29.962697 31927 ts_tablet_manager.cc:1397] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260805 22:42:29.964643 31927 raft_consensus.cc:348] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } }
I20260805 22:42:29.964825 31927 raft_consensus.cc:374] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:29.964881 31927 raft_consensus.cc:729] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be3c756baef4228b76d483e91715083, State: Initialized, Role: FOLLOWER
I20260805 22:42:29.964998 31927 consensus_queue.cc:260] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } }
I20260805 22:42:29.965274 31927 ts_tablet_manager.cc:1428] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:42:29.965337 31700 heartbeater.cc:503] Master 127.23.197.126:34675 was elected leader, sending a full tablet report...
I20260805 22:42:30.002490 31938 raft_consensus.cc:482] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:30.002624 31938 raft_consensus.cc:504] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } }
I20260805 22:42:30.003001 31938 leader_election.cc:288] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bef53706199546a690a9e7cdfe488d38 (127.23.197.67:36645), a419491a3ed0439187220e228c592fda (127.23.197.65:45237)
I20260805 22:42:30.004568 31916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.003341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:42875 (local address 127.23.197.67:36645)
0805 22:42:30.003402 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.003404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.003589 (+ 185us) server_negotiation.cc:407] Connection header received
0805 22:42:30.003656 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.003657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.003664 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.003758 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:30.004308 (+ 550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:30.004310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:30.004310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:30.004315 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:30.004333 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:30.004338 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:30.004402 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:30.004458 (+ 56us) server_negotiation.cc:299] Negotiation successful
0805 22:42:30.004475 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:42:30.004556 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.003307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:45301 (local address 127.23.197.65:45237)
0805 22:42:30.003353 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.003355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.003366 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:30.003443 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.003444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.003451 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.003532 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:30.004228 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:30.004229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:30.004230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:30.004237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:30.004281 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:30.004289 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:30.004368 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:30.004464 (+ 96us) server_negotiation.cc:299] Negotiation successful
0805 22:42:30.004480 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260805 22:42:30.004554 31702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.003242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:45237 (local address 127.23.197.66:45301)
0805 22:42:30.003301 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:30.003313 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:30.003357 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:30.003540 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:30.003541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:30.003544 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:30.004043 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:42:30.004044 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:30.004051 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:42:30.004081 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:30.004368 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:30.004386 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:30.004387 (+ 1us) client_negotiation.cc:769] Sending connection context
0805 22:42:30.004446 (+ 59us) client_negotiation.cc:240] Negotiation successful
0805 22:42:30.004462 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260805 22:42:30.004565 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.003255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:36645 (local address 127.23.197.66:42875)
0805 22:42:30.003492 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:30.003502 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:30.003590 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:30.003777 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:30.003778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:30.003780 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:30.004190 (+ 410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:42:30.004191 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:30.004196 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:42:30.004206 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:30.004406 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:30.004406 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:30.004407 (+ 1us) client_negotiation.cc:769] Sending connection context
0805 22:42:30.004456 (+ 49us) client_negotiation.cc:240] Negotiation successful
0805 22:42:30.004468 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":140,"threads_started":1}
I20260805 22:42:30.005033 31422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0088c8f9763492da1e926f19223f652" candidate_uuid: "0be3c756baef4228b76d483e91715083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a419491a3ed0439187220e228c592fda" is_pre_election: true
I20260805 22:42:30.005250 31422 raft_consensus.cc:2454] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0be3c756baef4228b76d483e91715083 in term 0.
I20260805 22:42:30.005221 31834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0088c8f9763492da1e926f19223f652" candidate_uuid: "0be3c756baef4228b76d483e91715083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef53706199546a690a9e7cdfe488d38" is_pre_election: true
I20260805 22:42:30.005390 31834 raft_consensus.cc:2454] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0be3c756baef4228b76d483e91715083 in term 0.
I20260805 22:42:30.005546 31518 leader_election.cc:302] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be3c756baef4228b76d483e91715083, a419491a3ed0439187220e228c592fda; no voters:
I20260805 22:42:30.005702 31938 raft_consensus.cc:2781] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:30.005811 31938 raft_consensus.cc:482] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:30.005853 31938 raft_consensus.cc:3037] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:30.006688 31938 raft_consensus.cc:504] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } }
I20260805 22:42:30.006870 31938 leader_election.cc:288] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [CANDIDATE]: Term 1 election: Requested vote from peers bef53706199546a690a9e7cdfe488d38 (127.23.197.67:36645), a419491a3ed0439187220e228c592fda (127.23.197.65:45237)
I20260805 22:42:30.007046 31422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0088c8f9763492da1e926f19223f652" candidate_uuid: "0be3c756baef4228b76d483e91715083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a419491a3ed0439187220e228c592fda"
I20260805 22:42:30.007087 31834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0088c8f9763492da1e926f19223f652" candidate_uuid: "0be3c756baef4228b76d483e91715083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bef53706199546a690a9e7cdfe488d38"
I20260805 22:42:30.007140 31422 raft_consensus.cc:3037] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:30.007217 31834 raft_consensus.cc:3037] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:30.008191 31422 raft_consensus.cc:2454] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0be3c756baef4228b76d483e91715083 in term 1.
I20260805 22:42:30.008320 31834 raft_consensus.cc:2454] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0be3c756baef4228b76d483e91715083 in term 1.
I20260805 22:42:30.008419 31518 leader_election.cc:302] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be3c756baef4228b76d483e91715083, a419491a3ed0439187220e228c592fda; no voters:
I20260805 22:42:30.008533 31938 raft_consensus.cc:2781] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:30.008739 31938 raft_consensus.cc:686] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 1 LEADER]: Becoming Leader. State: Replica: 0be3c756baef4228b76d483e91715083, State: Running, Role: LEADER
I20260805 22:42:30.008893 31938 consensus_queue.cc:237] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } }
I20260805 22:42:30.009856 31217 catalog_manager.cc:5495] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0be3c756baef4228b76d483e91715083 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "0be3c756baef4228b76d483e91715083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0be3c756baef4228b76d483e91715083" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:30.026135 31919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.025091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44310 (local address 127.23.197.66:43343)
0805 22:42:30.025135 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.025137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.025145 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:30.025256 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.025256 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.025270 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.025320 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:30.025909 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:30.025909 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:30.025910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:30.025912 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:30.025923 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:30.025927 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:30.025968 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:30.026013 (+ 45us) server_negotiation.cc:299] Negotiation successful
0805 22:42:30.026024 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:42:30.028046 31422 raft_consensus.cc:1261] T d0088c8f9763492da1e926f19223f652 P a419491a3ed0439187220e228c592fda [term 1 FOLLOWER]: Refusing update from remote peer 0be3c756baef4228b76d483e91715083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:30.028430 31943 consensus_queue.cc:1035] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a419491a3ed0439187220e228c592fda" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:30.028506 31834 raft_consensus.cc:1261] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [term 1 FOLLOWER]: Refusing update from remote peer 0be3c756baef4228b76d483e91715083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:30.028901 31943 consensus_queue.cc:1035] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bef53706199546a690a9e7cdfe488d38" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 36645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:30.031574 31951 mvcc.cc:205] Tried to move back new op lower bound from 7315332096112033792 to 7315332096037355520. Current Snapshot: MvccSnapshot[applied={T|T < 7315332096112033792}]
I20260805 22:42:30.032117 31954 mvcc.cc:205] Tried to move back new op lower bound from 7315332096112033792 to 7315332096037355520. Current Snapshot: MvccSnapshot[applied={T|T < 7315332096112033792}]
I20260805 22:42:30.033375 31217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:30.033473 31217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:30.034272 31217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 686a68e5561042b28ce7b7f7843f9208: Sending DeleteTablet for 3 replicas of tablet d0088c8f9763492da1e926f19223f652
I20260805 22:42:30.034590 31806 tablet_service.cc:1514] Processing DeleteTablet for tablet d0088c8f9763492da1e926f19223f652 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:30 UTC) from {username='slave'} at 127.0.0.1:44112
I20260805 22:42:30.034621 31393 tablet_service.cc:1514] Processing DeleteTablet for tablet d0088c8f9763492da1e926f19223f652 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:30 UTC) from {username='slave'} at 127.0.0.1:51014
I20260805 22:42:30.034857 31963 tablet_replica.cc:330] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: stopping tablet replica
I20260805 22:42:30.034859 31592 tablet_service.cc:1514] Processing DeleteTablet for tablet d0088c8f9763492da1e926f19223f652 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:30 UTC) from {username='slave'} at 127.0.0.1:44302
I20260805 22:42:30.034960 31963 raft_consensus.cc:2229] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:30.035050 31967 tablet_replica.cc:330] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: stopping tablet replica
I20260805 22:42:30.035066 31963 raft_consensus.cc:2258] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:30.035125 31967 raft_consensus.cc:2229] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:30.035121 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 31308
I20260805 22:42:30.035214 31967 raft_consensus.cc:2258] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:30.035624 31967 ts_tablet_manager.cc:1905] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:30.037415 31967 ts_tablet_manager.cc:1918] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:30.037475 31967 log.cc:1196] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/d0088c8f9763492da1e926f19223f652
I20260805 22:42:30.037706 31967 ts_tablet_manager.cc:1939] T d0088c8f9763492da1e926f19223f652 P 0be3c756baef4228b76d483e91715083: Deleting consensus metadata
I20260805 22:42:30.038143 31963 ts_tablet_manager.cc:1905] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:30.039397 31963 ts_tablet_manager.cc:1918] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:30.039448 31963 log.cc:1196] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/d0088c8f9763492da1e926f19223f652
I20260805 22:42:30.039630 31963 ts_tablet_manager.cc:1939] T d0088c8f9763492da1e926f19223f652 P bef53706199546a690a9e7cdfe488d38: Deleting consensus metadata
I20260805 22:42:30.039990 31193 catalog_manager.cc:4841] TS bef53706199546a690a9e7cdfe488d38 (127.23.197.67:36645): tablet d0088c8f9763492da1e926f19223f652 (table test-table [id=41a6f705c50b41ecbd9f71a709e0f4ba]) successfully deleted
I20260805 22:42:30.041141 31189 catalog_manager.cc:4841] TS 0be3c756baef4228b76d483e91715083 (127.23.197.66:43343): tablet d0088c8f9763492da1e926f19223f652 (table test-table [id=41a6f705c50b41ecbd9f71a709e0f4ba]) successfully deleted
W20260805 22:42:30.048101 31190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:45237 (error 108)
I20260805 22:42:30.048337 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 31495
I20260805 22:42:30.057390 31914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.057207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:45237 (local address 127.0.0.1:51020)
0805 22:42:30.057280 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:30.057346 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260805 22:42:30.057559 31190 catalog_manager.cc:4584] TS a419491a3ed0439187220e228c592fda (127.23.197.65:45237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0088c8f9763492da1e926f19223f652: Network error: Client connection negotiation failed: client connection to 127.23.197.65:45237: connect: Connection refused (error 111)
I20260805 22:42:30.061028 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 31703
I20260805 22:42:30.066952 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 31158
2026-08-05T22:42:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (704 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260805 22:42:30.090939 24341 mini_oidc.cc:152] Starting JWKS server
I20260805 22:42:30.092079 24341 webserver.cc:460] Webserver started at https://127.0.0.1:36605/ using document root <none> and password file <none>
I20260805 22:42:30.092216 24341 mini_oidc.cc:189] Starting OIDC Discovery server
I20260805 22:42:30.092414 24341 webserver.cc:460] Webserver started at http://127.0.0.1:44051/ using document root <none> and password file <none>
2026-08-05T22:42:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:30Z Disabled control of system clock
I20260805 22:42:30.101362 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36675
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:36675
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36605/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/testchainca.cert with env {}
W20260805 22:42:30.184571 31980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:30.184855 31980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:30.184892 31980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:30.186020 31980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:30.186059 31980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:30.186072 31980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:30.186085 31980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:30.186097 31980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 22:42:30.186110 31980 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 22:42:30.186131 31980 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36605/jwks.json
I20260805 22:42:30.187618 31980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:36675
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36605/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:30.187816 31980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:30.187978 31980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:30.192840 31988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:30.192975 31980 server_base.cc:1034] running on GCE node
W20260805 22:42:30.192854 31987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:30.193094 31992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:30.193368 31980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:30.193666 31980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:30.194841 31980 hybrid_clock.cc:648] HybridClock initialized: now 1785969750194823 us; error 41 us; skew 500 ppm
I20260805 22:42:30.196049 31980 webserver.cc:460] Webserver started at http://127.23.197.126:46835/ using document root <none> and password file <none>
I20260805 22:42:30.196295 31980 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:30.196346 31980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:30.196496 31980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:30.197607 31980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "0bfd84781503496c9d70b26d9615e21b"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "275af1c34b36abee58d0bfcddd10b5ad"
server_key_iv: "38d4f7032f8909ff977bfd4989846415"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.197997 31980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "0bfd84781503496c9d70b26d9615e21b"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "275af1c34b36abee58d0bfcddd10b5ad"
server_key_iv: "38d4f7032f8909ff977bfd4989846415"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.199337 31980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:42:30.200163 32000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:30.200397 31980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:30.200476 31980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "0bfd84781503496c9d70b26d9615e21b"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "275af1c34b36abee58d0bfcddd10b5ad"
server_key_iv: "38d4f7032f8909ff977bfd4989846415"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.200570 31980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:30.210079 31980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:30.210418 31980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:30.210585 31980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:30.214738 31980 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:36675
I20260805 22:42:30.214777 32076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:36675 every 8 connection(s)
I20260805 22:42:30.215139 31980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:30.215919 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 31980
I20260805 22:42:30.216024 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:30.216218 24341 external_mini_cluster.cc:1428] Setting key 0d70dbe9611c81c472fa95e7f73a9f87
I20260805 22:42:30.216244 32078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:30.219901 32078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b: Bootstrap starting.
I20260805 22:42:30.220630 32078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:30.221033 32078 log.cc:824] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:30.221899 32078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b: No bootstrap required, opened a new log
I20260805 22:42:30.222733 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.216828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.23.197.126:36675)
0805 22:42:30.217056 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.217066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.217097 (+ 31us) server_negotiation.cc:407] Connection header received
0805 22:42:30.217570 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.217580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.217667 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.217821 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:30.218020 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.219396 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.220265 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.220995 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.221120 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.221991 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:30.221995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:30.221995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:30.222211 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:30.222252 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:30.222265 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:30.222363 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:30.222423 (+ 60us) server_negotiation.cc:299] Negotiation successful
0805 22:42:30.222505 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260805 22:42:30.223867 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:42:30.224496 32078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfd84781503496c9d70b26d9615e21b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36675 } }
I20260805 22:42:30.224685 32078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:30.224725 32078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bfd84781503496c9d70b26d9615e21b, State: Initialized, Role: FOLLOWER
I20260805 22:42:30.224908 32078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfd84781503496c9d70b26d9615e21b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36675 } }
I20260805 22:42:30.225023 32078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:30.225057 32078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:30.225097 32078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:30.225685 32078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfd84781503496c9d70b26d9615e21b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36675 } }
I20260805 22:42:30.225775 32078 leader_election.cc:302] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bfd84781503496c9d70b26d9615e21b; no voters:
I20260805 22:42:30.225916 32078 leader_election.cc:288] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:30.226011 32084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:30.226177 32078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:30.226214 32084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [term 1 LEADER]: Becoming Leader. State: Replica: 0bfd84781503496c9d70b26d9615e21b, State: Running, Role: LEADER
I20260805 22:42:30.226361 32084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfd84781503496c9d70b26d9615e21b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36675 } }
I20260805 22:42:30.226800 32086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bfd84781503496c9d70b26d9615e21b. Latest consensus state: current_term: 1 leader_uuid: "0bfd84781503496c9d70b26d9615e21b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfd84781503496c9d70b26d9615e21b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36675 } } }
I20260805 22:42:30.226899 32086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:30.227154 32095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:30.227262 32085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bfd84781503496c9d70b26d9615e21b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfd84781503496c9d70b26d9615e21b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36675 } } }
I20260805 22:42:30.227366 32085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:30.227664 32095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:30.229357 32095 catalog_manager.cc:1347] Generated new cluster ID: 211e3b960b384ed494856ec99344e383
I20260805 22:42:30.229434 32095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:30.235915 32095 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:30.236435 32095 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:30.243496 32095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bfd84781503496c9d70b26d9615e21b: Generated new TSK 0
I20260805 22:42:30.243674 32095 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260805 22:42:30.273679 32122 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260805 22:42:30.273906 32122 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260805 22:42:30.274428 32121 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:30.256245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36675 (local address 127.0.0.1:37416)
0805 22:42:30.256434 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:30.256444 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:30.256505 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:30.256680 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:30.256681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:30.256683 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0805 22:42:30.256752 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:30.256754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.257735 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.257736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:30.258341 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:30.258343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.258831 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.258832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:30.258880 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.258899 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 22:42:30.273877 (+ 14978us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0805 22:42:30.273985 (+ 108us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.197.126:36675: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260805 22:42:30.274780 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.256338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37416 (local address 127.23.197.126:36675)
0805 22:42:30.256388 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.256390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.256508 (+ 118us) server_negotiation.cc:407] Connection header received
0805 22:42:30.256585 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.256586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.256598 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.256680 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0805 22:42:30.256847 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.257618 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.258426 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.258736 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.258806 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.259002 (+ 196us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 22:42:30.273708 (+ 14706us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0805 22:42:30.274680 (+ 972us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37416: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260805 22:42:30.274907 32082 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37416: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.23.197.126:36675: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36605/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260805 22:42:30.274991 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 31980
2026-08-05T22:42:30Z chronyd exiting
I20260805 22:42:30.319344 24341 test_util.cc:175] -----------------------------------------------
I20260805 22:42:30.319432 24341 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785969725087118-24341-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (229 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260805 22:42:30.320170 24341 mini_oidc.cc:152] Starting JWKS server
I20260805 22:42:30.320904 24341 webserver.cc:460] Webserver started at https://127.0.0.1:46811/ using document root <none> and password file <none>
I20260805 22:42:30.320982 24341 mini_oidc.cc:189] Starting OIDC Discovery server
I20260805 22:42:30.321141 24341 webserver.cc:460] Webserver started at http://127.0.0.1:46377/ using document root <none> and password file <none>
2026-08-05T22:42:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:30Z Disabled control of system clock
I20260805 22:42:30.330653 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41099
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:44865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:41099
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46811/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/testchainca.cert with env {}
W20260805 22:42:30.419032 32129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:30.419306 32129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:30.419351 32129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:30.420950 32129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:30.421015 32129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:30.421038 32129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:30.421087 32129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:30.421118 32129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 22:42:30.421173 32129 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 22:42:30.421209 32129 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46811/jwks.json
I20260805 22:42:30.422973 32129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:41099
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46811/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:30.423355 32129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:30.423605 32129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:30.428993 32140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:30.428999 32144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:30.429179 32129 server_base.cc:1034] running on GCE node
W20260805 22:42:30.429205 32139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:30.429651 32129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:30.430002 32129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:30.431200 32129 hybrid_clock.cc:648] HybridClock initialized: now 1785969750431185 us; error 34 us; skew 500 ppm
I20260805 22:42:30.432830 32129 webserver.cc:460] Webserver started at http://127.23.197.126:35319/ using document root <none> and password file <none>
I20260805 22:42:30.433107 32129 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:30.433184 32129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:30.433331 32129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:30.434564 32129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "71adf746def34d97923f0f9f0578849b"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "e51e9565342e427348603cacc00fa8b5"
server_key_iv: "8e6b3b3258f42c83be78f292feae6e66"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.434998 32129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "71adf746def34d97923f0f9f0578849b"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "e51e9565342e427348603cacc00fa8b5"
server_key_iv: "8e6b3b3258f42c83be78f292feae6e66"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.436493 32129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260805 22:42:30.437556 32151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:30.437812 32129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:30.437922 32129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "71adf746def34d97923f0f9f0578849b"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "e51e9565342e427348603cacc00fa8b5"
server_key_iv: "8e6b3b3258f42c83be78f292feae6e66"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.438014 32129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:30.472448 32129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:30.472987 32129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:30.473237 32129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:30.478099 32129 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:41099
I20260805 22:42:30.478174 32229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:41099 every 8 connection(s)
I20260805 22:42:30.478550 32129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:30.479447 32231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:30.484058 32231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b: Bootstrap starting.
I20260805 22:42:30.484915 32231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:30.485289 32231 log.cc:824] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:30.486192 32231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b: No bootstrap required, opened a new log
I20260805 22:42:30.487942 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 32129
I20260805 22:42:30.488076 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:30.488304 24341 external_mini_cluster.cc:1428] Setting key cf34bf4f1e046859624a1686ea25829f
I20260805 22:42:30.489114 32231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71adf746def34d97923f0f9f0578849b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41099 } }
I20260805 22:42:30.489328 32231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:30.489398 32231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71adf746def34d97923f0f9f0578849b, State: Initialized, Role: FOLLOWER
I20260805 22:42:30.489552 32231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71adf746def34d97923f0f9f0578849b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41099 } }
I20260805 22:42:30.489692 32231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:30.489749 32231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:30.489810 32231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:30.490408 32231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71adf746def34d97923f0f9f0578849b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41099 } }
I20260805 22:42:30.490540 32231 leader_election.cc:302] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71adf746def34d97923f0f9f0578849b; no voters:
I20260805 22:42:30.490732 32231 leader_election.cc:288] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:30.490757 32237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:30.490957 32237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [term 1 LEADER]: Becoming Leader. State: Replica: 71adf746def34d97923f0f9f0578849b, State: Running, Role: LEADER
I20260805 22:42:30.491086 32237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71adf746def34d97923f0f9f0578849b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41099 } }
I20260805 22:42:30.491086 32231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:30.491523 32242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71adf746def34d97923f0f9f0578849b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71adf746def34d97923f0f9f0578849b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41099 } } }
I20260805 22:42:30.491604 32242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:30.491767 32239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71adf746def34d97923f0f9f0578849b. Latest consensus state: current_term: 1 leader_uuid: "71adf746def34d97923f0f9f0578849b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71adf746def34d97923f0f9f0578849b" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41099 } } }
I20260805 22:42:30.491827 32239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:30.492262 32246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:30.492892 32246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:30.494802 32246 catalog_manager.cc:1347] Generated new cluster ID: a8f0c7bf89d347f49afdd6970cfd5d2b
I20260805 22:42:30.494855 32246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:30.495659 32236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.488875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.23.197.126:41099)
0805 22:42:30.489115 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.489126 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.489165 (+ 39us) server_negotiation.cc:407] Connection header received
0805 22:42:30.489298 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.489306 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.489385 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.489540 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:30.489827 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.491704 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.492973 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.493697 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.494181 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.494705 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:30.494710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:30.494711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:30.494958 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:30.495004 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:30.495021 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:30.495086 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:30.495316 (+ 230us) server_negotiation.cc:299] Negotiation successful
0805 22:42:30.495407 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260805 22:42:30.496559 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:42:30.510517 32246 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:30.511250 32246 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:30.522967 32246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71adf746def34d97923f0f9f0578849b: Generated new TSK 0
I20260805 22:42:30.523206 32246 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260805 22:42:30.554432 32274 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260805 22:42:30.554522 32274 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260805 22:42:30.554869 32236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.532463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53036 (local address 127.23.197.126:41099)
0805 22:42:30.532524 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.532527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.532674 (+ 147us) server_negotiation.cc:407] Connection header received
0805 22:42:30.532756 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.532756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.532825 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.532898 (+ 73us) server_negotiation.cc:226] Negotiated authn=JWT
0805 22:42:30.533058 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.533659 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.534794 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.534994 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.535039 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.535242 (+ 203us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 22:42:30.554607 (+ 19365us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0805 22:42:30.554811 (+ 204us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53036: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":34}
W20260805 22:42:30.554945 32236 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53036: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260805 22:42:30.555009 32273 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:30.532375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41099 (local address 127.0.0.1:53036)
0805 22:42:30.532594 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:30.532607 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:30.532676 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:30.532917 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:30.532919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:30.532921 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0805 22:42:30.532990 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:30.532992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.533748 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.533749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:30.534692 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:30.534696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.535064 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.535065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:30.535129 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.535134 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 22:42:30.554787 (+ 19653us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0805 22:42:30.554909 (+ 122us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.197.126:41099: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":100,"threads_started":1}
I20260805 22:42:30.555403 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 32129
2026-08-05T22:42:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (280 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260805 22:42:30.601358 24341 mini_oidc.cc:152] Starting JWKS server
I20260805 22:42:30.602113 24341 webserver.cc:460] Webserver started at https://127.0.0.1:38539/ using document root <none> and password file <none>
I20260805 22:42:30.602159 24341 mini_oidc.cc:189] Starting OIDC Discovery server
I20260805 22:42:30.602337 24341 webserver.cc:460] Webserver started at http://127.0.0.1:34483/ using document root <none> and password file <none>
2026-08-05T22:42:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:30Z Disabled control of system clock
I20260805 22:42:30.611701 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:34167
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:37671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:34167
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38539/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260805 22:42:30.694237 32280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:30.694438 32280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:30.694460 32280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:30.695660 32280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:30.695704 32280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:30.695716 32280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:30.695729 32280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:30.695786 32280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 22:42:30.695804 32280 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 22:42:30.695834 32280 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38539/jwks.json
I20260805 22:42:30.697218 32280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:34167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:34167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38539/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:30.697468 32280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:30.697786 32280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:30.702755 32290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:30.702761 32294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:30.702770 32291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:30.703090 32280 server_base.cc:1034] running on GCE node
I20260805 22:42:30.703354 32280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:30.703650 32280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:30.704900 32280 hybrid_clock.cc:648] HybridClock initialized: now 1785969750704884 us; error 56 us; skew 500 ppm
I20260805 22:42:30.706352 32280 webserver.cc:460] Webserver started at http://127.23.197.126:40379/ using document root <none> and password file <none>
I20260805 22:42:30.706599 32280 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:30.706736 32280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:30.706898 32280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:30.708026 32280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "65a700fa12f2420c9356602e871c618f"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "f3024ea32f872f96ba1fd2e85693e1ea"
server_key_iv: "8ac66173a9c6f59a3163b7f75fe07b75"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.708400 32280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "65a700fa12f2420c9356602e871c618f"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "f3024ea32f872f96ba1fd2e85693e1ea"
server_key_iv: "8ac66173a9c6f59a3163b7f75fe07b75"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.709853 32280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:30.710619 32301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:30.710814 32280 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:30.710924 32280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "65a700fa12f2420c9356602e871c618f"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "f3024ea32f872f96ba1fd2e85693e1ea"
server_key_iv: "8ac66173a9c6f59a3163b7f75fe07b75"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.711002 32280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:30.734607 32280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:30.734997 32280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:30.735172 32280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:30.739559 32280 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:34167
I20260805 22:42:30.739679 32386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:34167 every 8 connection(s)
I20260805 22:42:30.740002 32280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:30.740931 32387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:30.743811 32387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f: Bootstrap starting.
I20260805 22:42:30.744458 32387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:30.744800 32387 log.cc:824] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:30.745539 32387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f: No bootstrap required, opened a new log
I20260805 22:42:30.746426 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 32280
I20260805 22:42:30.746520 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:30.746747 24341 external_mini_cluster.cc:1428] Setting key d928648905ad05bc9035f8c27cb9cbc0
I20260805 22:42:30.747665 32387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65a700fa12f2420c9356602e871c618f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34167 } }
I20260805 22:42:30.747833 32387 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:30.747879 32387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a700fa12f2420c9356602e871c618f, State: Initialized, Role: FOLLOWER
I20260805 22:42:30.748013 32387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65a700fa12f2420c9356602e871c618f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34167 } }
I20260805 22:42:30.748113 32387 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:30.748159 32387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:30.748209 32387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:30.748911 32387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65a700fa12f2420c9356602e871c618f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34167 } }
I20260805 22:42:30.749001 32387 leader_election.cc:302] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65a700fa12f2420c9356602e871c618f; no voters:
I20260805 22:42:30.749195 32387 leader_election.cc:288] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:30.749222 32394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:30.749328 32394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [term 1 LEADER]: Becoming Leader. State: Replica: 65a700fa12f2420c9356602e871c618f, State: Running, Role: LEADER
I20260805 22:42:30.749406 32394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65a700fa12f2420c9356602e871c618f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34167 } }
I20260805 22:42:30.749464 32387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:30.749715 32394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65a700fa12f2420c9356602e871c618f. Latest consensus state: current_term: 1 leader_uuid: "65a700fa12f2420c9356602e871c618f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65a700fa12f2420c9356602e871c618f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34167 } } }
I20260805 22:42:30.749795 32394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:30.749723 32398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65a700fa12f2420c9356602e871c618f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65a700fa12f2420c9356602e871c618f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34167 } } }
I20260805 22:42:30.749960 32398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:30.750006 32403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:30.750595 32403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:30.752267 32403 catalog_manager.cc:1347] Generated new cluster ID: a6ca6983866e4b52beb52f16866bab9d
I20260805 22:42:30.752324 32403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:30.753851 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.747302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57496 (local address 127.23.197.126:34167)
0805 22:42:30.747532 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.747540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.747566 (+ 26us) server_negotiation.cc:407] Connection header received
0805 22:42:30.747681 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.747686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.747742 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.747888 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:30.748078 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.749867 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.751123 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.751717 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.751883 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.753024 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:30.753028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:30.753028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:30.753237 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:30.753281 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:30.753295 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:30.753372 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:30.753439 (+ 67us) server_negotiation.cc:299] Negotiation successful
0805 22:42:30.753634 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260805 22:42:30.754865 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:42:30.757977 32403 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:30.758678 32403 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:30.760944 32403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65a700fa12f2420c9356602e871c618f: Generated new TSK 0
I20260805 22:42:30.761132 32403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:42:30.773255 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.769153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.23.197.126:34167)
0805 22:42:30.769214 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.769216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.769273 (+ 57us) server_negotiation.cc:407] Connection header received
0805 22:42:30.769389 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.769390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.769402 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.769473 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0805 22:42:30.769688 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.770450 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.771342 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.771667 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.771742 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.771946 (+ 204us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 22:42:30.773025 (+ 1079us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0805 22:42:30.773170 (+ 145us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57510: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":29}
W20260805 22:42:30.773375 32431 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:30.768987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:34167 (local address 127.0.0.1:57510)
0805 22:42:30.769186 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:30.769202 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:30.769296 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:30.769492 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:30.769494 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:30.769496 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0805 22:42:30.769561 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:30.769563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.770583 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.770584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:30.771242 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:30.771245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.771753 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.771754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:30.771830 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.771835 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 22:42:30.773134 (+ 1299us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0805 22:42:30.773263 (+ 129us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.197.126:34167: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":95,"threads_started":1}
I20260805 22:42:30.773571 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 32280
E20260805 22:42:30.792878 32433 webserver.cc:533] Webserver: sslize failed:
E20260805 22:42:30.792986 32433 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-05T22:42:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (218 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-05T22:42:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:30Z Disabled control of system clock
I20260805 22:42:30.829363 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:39809
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:36027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:39809
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/testchainca.cert with env {}
W20260805 22:42:30.913286 32441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:30.913491 32441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:30.913512 32441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:30.914669 32441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:30.914713 32441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:30.914726 32441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:30.914745 32441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:30.914758 32441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 22:42:30.914770 32441 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 22:42:30.914781 32441 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260805 22:42:30.916324 32441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:39809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:39809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:30.916576 32441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:30.916844 32441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:30.921772 32449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:30.921864 32451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:30.921901 32448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:30.921998 32441 server_base.cc:1034] running on GCE node
I20260805 22:42:30.922295 32441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:30.922600 32441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:30.923750 32441 hybrid_clock.cc:648] HybridClock initialized: now 1785969750923740 us; error 28 us; skew 500 ppm
I20260805 22:42:30.925038 32441 webserver.cc:460] Webserver started at http://127.23.197.126:46253/ using document root <none> and password file <none>
I20260805 22:42:30.925287 32441 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:30.925336 32441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:30.925498 32441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:30.926549 32441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "744695c74c0049ada4d3d4730e2ecc06"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "4870c1862724d45fd25f2af95ff650d4"
server_key_iv: "76e511e1faf2e20180f63be1e4f43551"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.926930 32441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "744695c74c0049ada4d3d4730e2ecc06"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "4870c1862724d45fd25f2af95ff650d4"
server_key_iv: "76e511e1faf2e20180f63be1e4f43551"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.928339 32441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:30.929260 32461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:30.929462 32441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:42:30.929559 32441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "744695c74c0049ada4d3d4730e2ecc06"
format_stamp: "Formatted at 2026-08-05 22:42:30 on dist-test-slave-g170"
server_key: "4870c1862724d45fd25f2af95ff650d4"
server_key_iv: "76e511e1faf2e20180f63be1e4f43551"
server_key_version: "encryptionkey@0"
I20260805 22:42:30.929629 32441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:30.947331 32441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:30.947692 32441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:30.947856 32441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:30.952416 32441 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:39809
I20260805 22:42:30.952458 32538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:39809 every 8 connection(s)
I20260805 22:42:30.952922 32441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:30.953985 32539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:30.954641 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 32441
I20260805 22:42:30.954757 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:30.954954 24341 external_mini_cluster.cc:1428] Setting key 625aebac0d0efe75f87500d375dc7afe
I20260805 22:42:30.957156 32539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06: Bootstrap starting.
I20260805 22:42:30.957887 32539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:30.958256 32539 log.cc:824] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:30.959108 32539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06: No bootstrap required, opened a new log
I20260805 22:42:30.961002 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.955545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.23.197.126:39809)
0805 22:42:30.955763 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.955771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.955797 (+ 26us) server_negotiation.cc:407] Connection header received
0805 22:42:30.956208 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.956214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.956275 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.956394 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:30.956648 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.957648 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.958719 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.959231 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.959340 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:30.960148 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:30.960152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:30.960152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:42:30.960372 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:30.960410 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:42:30.960423 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:30.960488 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:30.960596 (+ 108us) server_negotiation.cc:299] Negotiation successful
0805 22:42:30.960706 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260805 22:42:30.961699 32539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744695c74c0049ada4d3d4730e2ecc06" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39809 } }
I20260805 22:42:30.961901 32539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:30.961948 32539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744695c74c0049ada4d3d4730e2ecc06, State: Initialized, Role: FOLLOWER
I20260805 22:42:30.962080 32539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744695c74c0049ada4d3d4730e2ecc06" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39809 } }
I20260805 22:42:30.962186 32539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:30.962195 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:42:30.962234 32539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:30.962286 32539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:30.963052 32539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744695c74c0049ada4d3d4730e2ecc06" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39809 } }
I20260805 22:42:30.963181 32539 leader_election.cc:302] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 744695c74c0049ada4d3d4730e2ecc06; no voters:
I20260805 22:42:30.963366 32539 leader_election.cc:288] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:30.963418 32546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:30.963546 32546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [term 1 LEADER]: Becoming Leader. State: Replica: 744695c74c0049ada4d3d4730e2ecc06, State: Running, Role: LEADER
I20260805 22:42:30.963644 32539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:30.963629 32546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744695c74c0049ada4d3d4730e2ecc06" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39809 } }
I20260805 22:42:30.964048 32549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 744695c74c0049ada4d3d4730e2ecc06. Latest consensus state: current_term: 1 leader_uuid: "744695c74c0049ada4d3d4730e2ecc06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744695c74c0049ada4d3d4730e2ecc06" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39809 } } }
I20260805 22:42:30.964311 32549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:30.964223 32547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "744695c74c0049ada4d3d4730e2ecc06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744695c74c0049ada4d3d4730e2ecc06" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39809 } } }
I20260805 22:42:30.964558 32547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:30.964684 32561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:30.965281 32561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:30.969254 32561 catalog_manager.cc:1347] Generated new cluster ID: c5b642e4157d4907898c0352e646d168
I20260805 22:42:30.969331 32561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:30.981145 32561 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:30.982002 32561 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:30.988096 32561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 744695c74c0049ada4d3d4730e2ecc06: Generated new TSK 0
I20260805 22:42:30.988310 32561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:42:31.018227 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:30.998116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.23.197.126:39809)
0805 22:42:30.998188 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:30.998191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:30.998341 (+ 150us) server_negotiation.cc:407] Connection header received
0805 22:42:30.998434 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:30.998435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:30.998449 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:30.998523 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0805 22:42:30.998670 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.999484 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:31.000322 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:31.000644 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:31.000716 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:31.001006 (+ 290us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 22:42:31.017932 (+ 16926us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0805 22:42:31.018144 (+ 212us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56000: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260805 22:42:31.018335 32543 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56000: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260805 22:42:31.018412 32579 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:30.998010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39809 (local address 127.0.0.1:56000)
0805 22:42:30.998233 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:30.998268 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:30.998345 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:30.998537 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:30.998538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:30.998540 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0805 22:42:30.998603 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:30.998606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:30.999592 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:30.999593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:31.000233 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:31.000235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:31.000812 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:31.000814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:31.000879 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:31.000885 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 22:42:31.018114 (+ 17229us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0805 22:42:31.018256 (+ 142us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.197.126:39809: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":144,"threads_started":1}
I20260805 22:42:31.019032 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 32441
2026-08-05T22:42:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (222 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (104 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (99 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:32 dist-test-slave-g170 krb5kdc[32601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:32 dist-test-slave-g170 krb5kdc[32601](info): set up 2 sockets
Aug 05 22:42:32 dist-test-slave-g170 krb5kdc[32601](info): commencing operation
krb5kdc: starting...
W20260805 22:42:34.168202 24341 mini_kdc.cc:121] Time spent starting KDC: real 2.922s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:34 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:34.308655 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40111
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:40111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:34.408499 32617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:34.408833 32617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:34.408883 32617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:34.410228 32617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:34.410275 32617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:34.410292 32617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:34.410306 32617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:34.410320 32617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:34.412006 32617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:40111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:34.412266 32617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:34.412483 32617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:34.418154 32625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:34.418123 32627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:34.418301 32617 server_base.cc:1034] running on GCE node
W20260805 22:42:34.418164 32629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:34.418625 32617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:34.418972 32617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:34.420159 32617 hybrid_clock.cc:648] HybridClock initialized: now 1785969754420141 us; error 49 us; skew 500 ppm
Aug 05 22:42:34 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969754, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:34.423121 32617 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:34.423663 32617 webserver.cc:460] Webserver started at http://127.23.197.126:45203/ using document root <none> and password file <none>
I20260805 22:42:34.423942 32617 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:34.424016 32617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:34.424175 32617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:34.425366 32617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "f5f2bf9569a44c9480b20d89333c0e6f"
format_stamp: "Formatted at 2026-08-05 22:42:34 on dist-test-slave-g170"
server_key: "e10a9baa877867adc1aeadde0358050b"
server_key_iv: "036c50903f0c1177f45f75a261267188"
server_key_version: "encryptionkey@0"
I20260805 22:42:34.425753 32617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "f5f2bf9569a44c9480b20d89333c0e6f"
format_stamp: "Formatted at 2026-08-05 22:42:34 on dist-test-slave-g170"
server_key: "e10a9baa877867adc1aeadde0358050b"
server_key_iv: "036c50903f0c1177f45f75a261267188"
server_key_version: "encryptionkey@0"
I20260805 22:42:34.427294 32617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:42:34.428339 32640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:34.428604 32617 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:34.428733 32617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "f5f2bf9569a44c9480b20d89333c0e6f"
format_stamp: "Formatted at 2026-08-05 22:42:34 on dist-test-slave-g170"
server_key: "e10a9baa877867adc1aeadde0358050b"
server_key_iv: "036c50903f0c1177f45f75a261267188"
server_key_version: "encryptionkey@0"
I20260805 22:42:34.428875 32617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:34.452726 32617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:34.455557 32617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:34.455799 32617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:34.460523 32617 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:40111
I20260805 22:42:34.460582 32708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:40111 every 8 connection(s)
I20260805 22:42:34.461014 32617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:34.461972 32710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:34.464298 32710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f: Bootstrap starting.
I20260805 22:42:34.464953 32710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:34.465233 32710 log.cc:824] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:34.465984 32710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f: No bootstrap required, opened a new log
I20260805 22:42:34.466741 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 32617
I20260805 22:42:34.466882 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:34.467100 24341 external_mini_cluster.cc:1428] Setting key cb20b180ad524d87eb8487f429722f21
I20260805 22:42:34.468246 32710 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f2bf9569a44c9480b20d89333c0e6f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40111 } }
I20260805 22:42:34.468427 32710 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:34.468468 32710 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f2bf9569a44c9480b20d89333c0e6f, State: Initialized, Role: FOLLOWER
I20260805 22:42:34.468619 32710 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f2bf9569a44c9480b20d89333c0e6f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40111 } }
I20260805 22:42:34.468701 32710 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:34.468725 32710 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:34.468818 32710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:34.469985 32710 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f2bf9569a44c9480b20d89333c0e6f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40111 } }
I20260805 22:42:34.470112 32710 leader_election.cc:302] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5f2bf9569a44c9480b20d89333c0e6f; no voters:
I20260805 22:42:34.470302 32710 leader_election.cc:288] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:34.470381 32717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:34.470553 32717 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [term 1 LEADER]: Becoming Leader. State: Replica: f5f2bf9569a44c9480b20d89333c0e6f, State: Running, Role: LEADER
I20260805 22:42:34.470572 32710 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:34.470654 32717 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f2bf9569a44c9480b20d89333c0e6f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40111 } }
I20260805 22:42:34.471226 32718 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5f2bf9569a44c9480b20d89333c0e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f2bf9569a44c9480b20d89333c0e6f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40111 } } }
I20260805 22:42:34.471217 32717 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5f2bf9569a44c9480b20d89333c0e6f. Latest consensus state: current_term: 1 leader_uuid: "f5f2bf9569a44c9480b20d89333c0e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f2bf9569a44c9480b20d89333c0e6f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40111 } } }
I20260805 22:42:34.471359 32717 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:34.471318 32718 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:34.471606 32726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:34.472205 32726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:34.474370 32726 catalog_manager.cc:1347] Generated new cluster ID: 941ddafb10e344ba9da0eba0f114305b
I20260805 22:42:34.474426 32726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:42:34 dist-test-slave-g170 krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:34.479256 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:34.467720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.23.197.126:40111)
0805 22:42:34.467948 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:34.467961 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:34.468000 (+ 39us) server_negotiation.cc:407] Connection header received
0805 22:42:34.468186 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:34.468195 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:34.468256 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:34.468412 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:34.469294 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.470797 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.472042 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.472885 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.473033 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:34.475780 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:34.475787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:34.475787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:34.475802 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:34.477555 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:34.477956 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:34.477960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:34.477960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:34.478069 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:34.478303 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:34.478305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:34.478305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:34.478563 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:34.478756 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:34.478828 (+ 72us) server_negotiation.cc:299] Negotiation successful
0805 22:42:34.478985 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
I20260805 22:42:34.516009 32726 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:34.516638 32726 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:34.521176 32726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5f2bf9569a44c9480b20d89333c0e6f: Generated new TSK 0
I20260805 22:42:34.521389 32726 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:34.560834 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40111
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:34.652419 32745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:34.652644 32745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:34.652670 32745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:34.653872 32745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:34.653925 32745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:34.653954 32745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:34.655412 32745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:34.655611 32745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:34.655802 32745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:34.661201 32756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:34.661209 32752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:34.661355 32745 server_base.cc:1034] running on GCE node
W20260805 22:42:34.661453 32754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:34.661705 32745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:34.661989 32745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:34.663173 32745 hybrid_clock.cc:648] HybridClock initialized: now 1785969754663163 us; error 43 us; skew 500 ppm
Aug 05 22:42:34 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969754, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:34.665970 32745 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:34.666448 32745 webserver.cc:460] Webserver started at http://127.23.197.65:44051/ using document root <none> and password file <none>
I20260805 22:42:34.666641 32745 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:34.666685 32745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:34.666785 32745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:34.667884 32745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "a0d404bbfaeb4ce1bbda947048d49fd0"
format_stamp: "Formatted at 2026-08-05 22:42:34 on dist-test-slave-g170"
server_key: "16338648bf30e8389b7ef9a34536e95a"
server_key_iv: "77edc30433ff442e7439da41bc36263f"
server_key_version: "encryptionkey@0"
I20260805 22:42:34.668211 32745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "a0d404bbfaeb4ce1bbda947048d49fd0"
format_stamp: "Formatted at 2026-08-05 22:42:34 on dist-test-slave-g170"
server_key: "16338648bf30e8389b7ef9a34536e95a"
server_key_iv: "77edc30433ff442e7439da41bc36263f"
server_key_version: "encryptionkey@0"
I20260805 22:42:34.669685 32745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260805 22:42:34.670506 32767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:34.670687 32745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:34.670796 32745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "a0d404bbfaeb4ce1bbda947048d49fd0"
format_stamp: "Formatted at 2026-08-05 22:42:34 on dist-test-slave-g170"
server_key: "16338648bf30e8389b7ef9a34536e95a"
server_key_iv: "77edc30433ff442e7439da41bc36263f"
server_key_version: "encryptionkey@0"
I20260805 22:42:34.670886 32745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:34.682005 32745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:34.684628 32745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:34.684849 32745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:34.685114 32745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:34.685442 32745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:34.685497 32745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:34.685542 32745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:34.685571 32745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:34.691663 32745 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:42333
I20260805 22:42:34.691708 456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:42333 every 8 connection(s)
I20260805 22:42:34.692040 32745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:34.697510 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 32745
I20260805 22:42:34.697657 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:34.697932 24341 external_mini_cluster.cc:1428] Setting key 3c19ac62951ac212b154d3896f1cc370
Aug 05 22:42:34 dist-test-slave-g170 krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969754, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:34.703298 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:34.692862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:36281 (local address 127.23.197.126:40111)
0805 22:42:34.692940 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:34.692943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:34.693159 (+ 216us) server_negotiation.cc:407] Connection header received
0805 22:42:34.694175 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:34.694176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:34.694190 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:34.694253 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:34.694932 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.695712 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.696834 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.697051 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.697144 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:34.699829 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:34.699831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:34.699832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:34.699840 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:34.701515 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:34.702112 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:34.702114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:34.702114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:34.702192 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:34.702520 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:34.702521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:34.702521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:34.702657 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:34.702724 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:34.703132 (+ 408us) server_negotiation.cc:299] Negotiation successful
0805 22:42:34.703226 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 22:42:34.703739 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:34.692825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40111 (local address 127.23.197.65:36281)
0805 22:42:34.693015 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:34.693066 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:34.694047 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:34.694326 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:34.694330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:34.694618 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:34.694818 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:34.694822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.695855 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.695856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:34.696697 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:34.696700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.697161 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.697162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:34.697359 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:34.698043 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:34.698055 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:34.699729 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:34.701629 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:34.701658 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:34.701661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:34.702023 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:34.702268 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:34.702269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:34.702270 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:34.702425 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:34.702745 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:34.702752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:34.702941 (+ 189us) client_negotiation.cc:769] Sending connection context
0805 22:42:34.703066 (+ 125us) client_negotiation.cc:240] Negotiation successful
0805 22:42:34.703307 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260805 22:42:34.704435 458 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40111
I20260805 22:42:34.704557 458 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:34.704819 458 heartbeater.cc:511] Master 127.23.197.126:40111 requested a full tablet report, sending...
I20260805 22:42:34.705492 32663 ts_manager.cc:194] Registered new tserver with Master: a0d404bbfaeb4ce1bbda947048d49fd0 (127.23.197.65:42333)
I20260805 22:42:34.706205 32663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:36281
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:34.728678 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40111
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:34.815338 464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:34.815583 464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:34.815635 464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:34.816836 464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:34.816905 464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:34.816948 464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:34.818516 464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:34.818807 464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:34.819056 464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:34.824462 472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:34.824517 475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:34.824570 471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:34.824898 464 server_base.cc:1034] running on GCE node
I20260805 22:42:34.825098 464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:34.825387 464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:34.826561 464 hybrid_clock.cc:648] HybridClock initialized: now 1785969754826530 us; error 49 us; skew 500 ppm
Aug 05 22:42:34 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969754, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:34.829592 464 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:34.830123 464 webserver.cc:460] Webserver started at http://127.23.197.66:36515/ using document root <none> and password file <none>
I20260805 22:42:34.830369 464 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:34.830417 464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:34.830581 464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:34.831779 464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "0608b0f3893640009ffadb01c71a6298"
format_stamp: "Formatted at 2026-08-05 22:42:34 on dist-test-slave-g170"
server_key: "f4f58ae6894776cd07d1a03e2bcb6cc3"
server_key_iv: "f4ecff5f3ae67bf454e5928eb7892c6a"
server_key_version: "encryptionkey@0"
I20260805 22:42:34.832190 464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "0608b0f3893640009ffadb01c71a6298"
format_stamp: "Formatted at 2026-08-05 22:42:34 on dist-test-slave-g170"
server_key: "f4f58ae6894776cd07d1a03e2bcb6cc3"
server_key_iv: "f4ecff5f3ae67bf454e5928eb7892c6a"
server_key_version: "encryptionkey@0"
I20260805 22:42:34.834043 464 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260805 22:42:34.835048 483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:34.835255 464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:34.835377 464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "0608b0f3893640009ffadb01c71a6298"
format_stamp: "Formatted at 2026-08-05 22:42:34 on dist-test-slave-g170"
server_key: "f4f58ae6894776cd07d1a03e2bcb6cc3"
server_key_iv: "f4ecff5f3ae67bf454e5928eb7892c6a"
server_key_version: "encryptionkey@0"
I20260805 22:42:34.835460 464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:34.860257 464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:34.862906 464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:34.863098 464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:34.863379 464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:34.863719 464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:34.863776 464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:34.863840 464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:34.863881 464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:34.873613 464 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:42533
I20260805 22:42:34.873664 641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:42533 every 8 connection(s)
I20260805 22:42:34.874109 464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:34.875425 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 464
I20260805 22:42:34.875528 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:34.875794 24341 external_mini_cluster.cc:1428] Setting key dedfa0cca36d5ce72dfb8a1401e146e9
Aug 05 22:42:34 dist-test-slave-g170 krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969754, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
I20260805 22:42:34.886586 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:34.874912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:40125 (local address 127.23.197.126:40111)
0805 22:42:34.874968 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:34.874970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:34.875255 (+ 285us) server_negotiation.cc:407] Connection header received
0805 22:42:34.876389 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:34.876390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:34.876403 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:34.876486 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:34.877335 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.878078 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.879744 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.879980 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.880063 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:34.883029 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:34.883030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:34.883031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:34.883041 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:34.884919 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:34.885401 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:34.885403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:34.885403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:34.885474 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:34.885761 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:34.885762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:34.885762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:34.885927 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:34.886016 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:34.886374 (+ 358us) server_negotiation.cc:299] Negotiation successful
0805 22:42:34.886476 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260805 22:42:34.886860 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:34.874861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40111 (local address 127.23.197.66:40125)
0805 22:42:34.875048 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:34.875095 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:34.876290 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:34.876587 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:34.876592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:34.877015 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:34.877225 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:34.877231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.878210 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.878211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:34.879556 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:34.879560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:34.880135 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:34.880137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:34.880263 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:34.880972 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:34.881004 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:34.882913 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:34.885033 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:34.885041 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:34.885043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:34.885304 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:34.885556 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:34.885557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:34.885557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:34.885673 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:34.886043 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:34.886050 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:34.886210 (+ 160us) client_negotiation.cc:769] Sending connection context
0805 22:42:34.886322 (+ 112us) client_negotiation.cc:240] Negotiation successful
0805 22:42:34.886539 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260805 22:42:34.887441 642 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40111
I20260805 22:42:34.887576 642 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:34.887843 642 heartbeater.cc:511] Master 127.23.197.126:40111 requested a full tablet report, sending...
I20260805 22:42:34.888325 32663 ts_manager.cc:194] Registered new tserver with Master: 0608b0f3893640009ffadb01c71a6298 (127.23.197.66:42533)
I20260805 22:42:34.888907 32663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:40125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:34.908563 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40111
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:34.996556 648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:34.996788 648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:34.996814 648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:34.997941 648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:34.997979 648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:34.998004 648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:34.999467 648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:34.999652 648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:34.999850 648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:35.005125 660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:35.005108 656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:35.005079 657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:35.005441 648 server_base.cc:1034] running on GCE node
I20260805 22:42:35.005673 648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:35.005950 648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:35.007123 648 hybrid_clock.cc:648] HybridClock initialized: now 1785969755007112 us; error 33 us; skew 500 ppm
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:35.009924 648 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:35.010471 648 webserver.cc:460] Webserver started at http://127.23.197.67:34795/ using document root <none> and password file <none>
I20260805 22:42:35.010746 648 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:35.010795 648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:35.010936 648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:35.012061 648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "f94cf6617f4743fab73c7a5824e6e900"
format_stamp: "Formatted at 2026-08-05 22:42:35 on dist-test-slave-g170"
server_key: "4a1461a53f64ff5536613292da5ecfbd"
server_key_iv: "793333ad86a2c3f0200947d8a34c6911"
server_key_version: "encryptionkey@0"
I20260805 22:42:35.012446 648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "f94cf6617f4743fab73c7a5824e6e900"
format_stamp: "Formatted at 2026-08-05 22:42:35 on dist-test-slave-g170"
server_key: "4a1461a53f64ff5536613292da5ecfbd"
server_key_iv: "793333ad86a2c3f0200947d8a34c6911"
server_key_version: "encryptionkey@0"
I20260805 22:42:35.014004 648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:35.014899 669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:35.015133 648 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:35.015254 648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "f94cf6617f4743fab73c7a5824e6e900"
format_stamp: "Formatted at 2026-08-05 22:42:35 on dist-test-slave-g170"
server_key: "4a1461a53f64ff5536613292da5ecfbd"
server_key_iv: "793333ad86a2c3f0200947d8a34c6911"
server_key_version: "encryptionkey@0"
I20260805 22:42:35.015331 648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:35.031599 648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:35.034324 648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:35.034497 648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:35.034790 648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:35.035154 648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:35.035211 648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:35.035272 648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:35.035303 648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:35.041922 648 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:46239
I20260805 22:42:35.041986 836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:46239 every 8 connection(s)
I20260805 22:42:35.042335 648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:35.045181 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 648
I20260805 22:42:35.045306 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:35.045503 24341 external_mini_cluster.cc:1428] Setting key 603e4b8f154ed57f1c4b18b8f074e597
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:35.052312 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:35.043168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:49185 (local address 127.23.197.126:40111)
0805 22:42:35.043241 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:35.043243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:35.043504 (+ 261us) server_negotiation.cc:407] Connection header received
0805 22:42:35.044539 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:35.044541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:35.044552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:35.044605 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:35.045317 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.045822 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.046847 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.047046 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.047135 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:35.049526 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:35.049528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:35.049528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:35.049536 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:35.050865 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:35.051292 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:35.051293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:35.051294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:35.051361 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:35.051619 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:35.051620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:35.051620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:35.051747 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:35.051821 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:35.052113 (+ 292us) server_negotiation.cc:299] Negotiation successful
0805 22:42:35.052208 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260805 22:42:35.052495 843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:35.043163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40111 (local address 127.23.197.67:49185)
0805 22:42:35.043373 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:35.043412 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:35.044438 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:35.044668 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:35.044672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:35.045014 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:35.045215 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:35.045219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.045941 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.045942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:35.046755 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:35.046757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.047148 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.047149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:35.047278 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:35.047829 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:35.047838 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:35.049425 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:35.050968 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:35.050976 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:35.050978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:35.051211 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:35.051432 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:35.051433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:35.051434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:35.051549 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:35.051824 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:35.051832 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:35.051974 (+ 142us) client_negotiation.cc:769] Sending connection context
0805 22:42:35.052078 (+ 104us) client_negotiation.cc:240] Negotiation successful
0805 22:42:35.052249 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":122,"threads_started":1}
I20260805 22:42:35.053175 838 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40111
I20260805 22:42:35.053268 838 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:35.053480 838 heartbeater.cc:511] Master 127.23.197.126:40111 requested a full tablet report, sending...
I20260805 22:42:35.053930 32663 ts_manager.cc:194] Registered new tserver with Master: f94cf6617f4743fab73c7a5824e6e900 (127.23.197.67:46239)
I20260805 22:42:35.054332 32663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:49185
I20260805 22:42:35.057444 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/test-admin.keytab.
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:35.074653 24341 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260805 22:42:35.077040 849 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:35.075857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.23.197.65:42333)
0805 22:42:35.076057 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:35.076062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:35.076136 (+ 74us) server_negotiation.cc:407] Connection header received
0805 22:42:35.076205 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:35.076207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:35.076223 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:35.076287 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:35.076915 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52372: BlockingRecv error: recv got EOF from 127.0.0.1:52372 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
W20260805 22:42:35.077064 24341 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5cc' of type FILE
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:35.078107 24341 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:35.079056 24341 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:35.087375 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:35.079605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.23.197.65:42333)
0805 22:42:35.079649 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:35.079651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:35.079736 (+ 85us) server_negotiation.cc:407] Connection header received
0805 22:42:35.079790 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:35.079791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:35.079802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:35.079855 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:35.080242 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.081193 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.081921 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.082262 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.082347 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:35.084539 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:35.084547 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:35.084547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:35.084555 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:35.086209 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:35.086532 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:35.086535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:35.086535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:35.086638 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:35.086817 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:35.086817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:35.086818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:35.086951 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:35.087069 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:35.087132 (+ 63us) server_negotiation.cc:299] Negotiation successful
0805 22:42:35.087201 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260805 22:42:35.089377 849 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:35.088696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.23.197.65:42333)
0805 22:42:35.088789 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:35.088791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:35.088874 (+ 83us) server_negotiation.cc:407] Connection header received
0805 22:42:35.088946 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:35.088947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:35.088959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:35.089026 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:35.089327 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52384: BlockingRecv error: recv got EOF from 127.0.0.1:52384 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260805 22:42:35.089493 24341 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5cc' of type FILE
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:35.090461 24341 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:35.091351 24341 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:35.098775 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:35.091882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52388 (local address 127.23.197.65:42333)
0805 22:42:35.091960 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:35.091962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:35.092041 (+ 79us) server_negotiation.cc:407] Connection header received
0805 22:42:35.092115 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:35.092116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:35.092130 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:35.092190 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:35.092575 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.093153 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.093851 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.094050 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.094134 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:35.096249 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:35.096251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:35.096251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:35.096258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:35.097619 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:35.097977 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:35.097979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:35.097979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:35.098037 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:35.098251 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:35.098253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:35.098253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:35.098367 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:35.098467 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:35.098580 (+ 113us) server_negotiation.cc:299] Negotiation successful
0805 22:42:35.098662 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260805 22:42:35.100443 849 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:35.099879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.23.197.65:42333)
0805 22:42:35.099936 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:35.099938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:35.099947 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:35.099969 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:35.099970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:35.099980 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:35.100047 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:35.100381 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52394: BlockingRecv error: recv got EOF from 127.0.0.1:52394 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260805 22:42:35.100503 24341 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785969725087118-24341-0/krb5kdc/krb5cc' of type FILE
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:35.101550 24341 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:35.102435 24341 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[32601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:42:35.109756 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:35.102975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52404 (local address 127.23.197.65:42333)
0805 22:42:35.103035 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:35.103037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:35.103063 (+ 26us) server_negotiation.cc:407] Connection header received
0805 22:42:35.103126 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:35.103126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:35.103133 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:35.103181 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:35.103674 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.104115 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.104985 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:35.105169 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:35.105216 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:35.107363 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:35.107364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:35.107364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:35.107371 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:35.108679 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:35.109093 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:35.109093 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:35.109094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:35.109147 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:35.109334 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:35.109335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:35.109335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:35.109425 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:35.109496 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:35.109607 (+ 111us) server_negotiation.cc:299] Negotiation successful
0805 22:42:35.109665 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260805 22:42:35.110421 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 32745
I20260805 22:42:35.121976 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 464
I20260805 22:42:35.139819 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 648
I20260805 22:42:35.152165 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 32617
2026-08-05T22:42:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3932 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[869](info): set up 2 sockets
Aug 05 22:42:35 dist-test-slave-g170 krb5kdc[869](info): commencing operation
krb5kdc: starting...
W20260805 22:42:36.391285 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:36 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.126.keytab.
Entry for principal oryx/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.126.keytab.
I20260805 22:42:36.524492 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:37131
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.126.keytab
--principal=oryx/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:37131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:36.609310 884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:36.609531 884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:36.609555 884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:36.610720 884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:36.610766 884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:36.610781 884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:36.610795 884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:36.610808 884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:36.612347 884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:37131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.126.keytab
--principal=oryx/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:36.612596 884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:36.612880 884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:36.618006 895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:36.617939 890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:36.617920 892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:36.618273 884 server_base.cc:1034] running on GCE node
I20260805 22:42:36.618549 884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:36.618880 884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:36.620044 884 hybrid_clock.cc:648] HybridClock initialized: now 1785969756620034 us; error 41 us; skew 500 ppm
Aug 05 22:42:36 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:36.622718 884 init.cc:377] Logged in from keytab as oryx/127.23.197.126@KRBTEST.COM (short username oryx)
I20260805 22:42:36.623275 884 webserver.cc:460] Webserver started at http://127.23.197.126:36547/ using document root <none> and password file <none>
I20260805 22:42:36.623515 884 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:36.623584 884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:36.623739 884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:36.624951 884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "c0aa186bced34b6d9599b9126c511cbd"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "9a01d89e778bc645e911168ecea827ce"
server_key_iv: "43e71b108eb0ac9eddaf5582b4312b4e"
server_key_version: "encryptionkey@0"
I20260805 22:42:36.625351 884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "c0aa186bced34b6d9599b9126c511cbd"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "9a01d89e778bc645e911168ecea827ce"
server_key_iv: "43e71b108eb0ac9eddaf5582b4312b4e"
server_key_version: "encryptionkey@0"
I20260805 22:42:36.626907 884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:42:36.627772 902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:36.627969 884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:36.628074 884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "c0aa186bced34b6d9599b9126c511cbd"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "9a01d89e778bc645e911168ecea827ce"
server_key_iv: "43e71b108eb0ac9eddaf5582b4312b4e"
server_key_version: "encryptionkey@0"
I20260805 22:42:36.628147 884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:36.646108 884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:36.648387 884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:36.648559 884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:36.653178 884 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:37131
I20260805 22:42:36.653244 982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:37131 every 8 connection(s)
I20260805 22:42:36.653640 884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:36.654680 983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:36.657119 983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: Bootstrap starting.
I20260805 22:42:36.657713 983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:36.658025 983 log.cc:824] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:36.658876 983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: No bootstrap required, opened a new log
I20260805 22:42:36.660182 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 884
I20260805 22:42:36.660308 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:36.660540 24341 external_mini_cluster.cc:1428] Setting key b02bf2b45da1ec6fc33b3ca4e4820de4
I20260805 22:42:36.660988 983 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } }
I20260805 22:42:36.661221 983 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:36.661303 983 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0aa186bced34b6d9599b9126c511cbd, State: Initialized, Role: FOLLOWER
I20260805 22:42:36.661446 983 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } }
I20260805 22:42:36.661528 983 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:36.661559 983 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:36.661617 983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:36.662729 983 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } }
I20260805 22:42:36.662843 983 leader_election.cc:302] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0aa186bced34b6d9599b9126c511cbd; no voters:
I20260805 22:42:36.662994 983 leader_election.cc:288] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:36.663046 988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:36.663214 988 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 1 LEADER]: Becoming Leader. State: Replica: c0aa186bced34b6d9599b9126c511cbd, State: Running, Role: LEADER
I20260805 22:42:36.663293 983 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:36.663435 988 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } }
I20260805 22:42:36.663910 988 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0aa186bced34b6d9599b9126c511cbd. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } }
I20260805 22:42:36.664021 988 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:36.664260 1002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:36.664829 1002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:36.664901 993 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } }
I20260805 22:42:36.664992 993 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:36.667423 1002 catalog_manager.cc:1347] Generated new cluster ID: 989888fdf71e43e4b1459dcdf7f54a1b
I20260805 22:42:36.667487 1002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:42:36 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:36.672982 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:36.661143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34158 (local address 127.23.197.126:37131)
0805 22:42:36.661339 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:36.661349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:36.661382 (+ 33us) server_negotiation.cc:407] Connection header received
0805 22:42:36.661500 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:36.661506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:36.661565 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:36.661727 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:36.662338 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:36.663878 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:36.665001 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:36.665599 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:36.665712 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:36.668486 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:36.668490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:36.668490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:36.668503 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:36.670703 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:36.671113 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:36.671117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:36.671117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:36.671257 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:36.671465 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:36.671465 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:36.671466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:36.671815 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:36.672511 (+ 696us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:36.672587 (+ 76us) server_negotiation.cc:299] Negotiation successful
0805 22:42:36.672745 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260805 22:42:36.681792 1002 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:36.682324 1002 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:36.689641 1002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: Generated new TSK 0
I20260805 22:42:36.689875 1002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab.
Entry for principal oryx/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab.
I20260805 22:42:36.724221 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:37131
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab
--principal=oryx/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:36.809844 1023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:36.810094 1023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:36.810135 1023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:36.811344 1023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:36.811414 1023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:36.811455 1023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:36.812965 1023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab
--principal=oryx/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:36.813242 1023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:36.813486 1023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:36.818977 1038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:36.818959 1031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:36.818964 1032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:36.819204 1023 server_base.cc:1034] running on GCE node
I20260805 22:42:36.819485 1023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:36.819826 1023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:36.820981 1023 hybrid_clock.cc:648] HybridClock initialized: now 1785969756820954 us; error 50 us; skew 500 ppm
Aug 05 22:42:36 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:36.823906 1023 init.cc:377] Logged in from keytab as oryx/127.23.197.65@KRBTEST.COM (short username oryx)
I20260805 22:42:36.824424 1023 webserver.cc:460] Webserver started at http://127.23.197.65:36663/ using document root <none> and password file <none>
I20260805 22:42:36.824680 1023 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:36.824781 1023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:36.824949 1023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:36.826125 1023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "0437f45cd9544ee0aacaeadd7a5b0f34"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "d736fd692ccc5aa389c312d5b647d501"
server_key_iv: "fd43a4f70a6ecd4061a60d0388a614cc"
server_key_version: "encryptionkey@0"
I20260805 22:42:36.826529 1023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "0437f45cd9544ee0aacaeadd7a5b0f34"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "d736fd692ccc5aa389c312d5b647d501"
server_key_iv: "fd43a4f70a6ecd4061a60d0388a614cc"
server_key_version: "encryptionkey@0"
I20260805 22:42:36.828033 1023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:36.828922 1048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:36.829109 1023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:36.829221 1023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "0437f45cd9544ee0aacaeadd7a5b0f34"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "d736fd692ccc5aa389c312d5b647d501"
server_key_iv: "fd43a4f70a6ecd4061a60d0388a614cc"
server_key_version: "encryptionkey@0"
I20260805 22:42:36.829308 1023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:36.853191 1023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:36.855626 1023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:36.855851 1023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:36.856129 1023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:36.856483 1023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:36.856541 1023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:36.856586 1023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:36.856632 1023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:36.863312 1023 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:33123
I20260805 22:42:36.863371 1231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:33123 every 8 connection(s)
I20260805 22:42:36.863853 1023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
Aug 05 22:42:36 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:36.870736 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 1023
I20260805 22:42:36.870880 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:36.871125 24341 external_mini_cluster.cc:1428] Setting key fd1cd74306e67089a3e938ff9c6dff2b
I20260805 22:42:36.874532 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:36.864731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:42135 (local address 127.23.197.126:37131)
0805 22:42:36.864818 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:36.864821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:36.865103 (+ 282us) server_negotiation.cc:407] Connection header received
0805 22:42:36.865651 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:36.865652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:36.865668 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:36.865730 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:36.866770 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:36.867292 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:36.868327 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:36.868532 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:36.868592 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:36.871066 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:36.871067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:36.871067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:36.871076 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:36.872752 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:36.873340 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:36.873341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:36.873342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:36.873402 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:36.873734 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:36.873735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:36.873736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:36.873900 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:36.873983 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:36.874304 (+ 321us) server_negotiation.cc:299] Negotiation successful
0805 22:42:36.874409 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 22:42:36.874578 1235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:36.864713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.65:42135)
0805 22:42:36.864945 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:36.864985 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:36.865523 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:36.865847 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:36.865850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:36.866441 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:36.866637 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:36.866641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:36.867386 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:36.867387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:36.868246 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:36.868248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:36.868636 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:36.868637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:36.868751 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:36.869369 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:36.869379 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:36.870955 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:36.872893 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:36.872902 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:36.872904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:36.873247 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:36.873473 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:36.873474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:36.873475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:36.873639 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:36.874018 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:36.874026 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:36.874155 (+ 129us) client_negotiation.cc:769] Sending connection context
0805 22:42:36.874260 (+ 105us) client_negotiation.cc:240] Negotiation successful
0805 22:42:36.874397 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":129,"threads_started":1}
I20260805 22:42:36.875175 1232 heartbeater.cc:347] Connected to a master server at 127.23.197.126:37131
I20260805 22:42:36.875324 1232 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:36.875511 1232 heartbeater.cc:511] Master 127.23.197.126:37131 requested a full tablet report, sending...
I20260805 22:42:36.876051 930 ts_manager.cc:194] Registered new tserver with Master: 0437f45cd9544ee0aacaeadd7a5b0f34 (127.23.197.65:33123)
I20260805 22:42:36.876744 930 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:42135
WARNING: no policy specified for oryx/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab.
Entry for principal oryx/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab.
I20260805 22:42:36.900308 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:37131
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab
--principal=oryx/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:36.985842 1239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:36.986057 1239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:36.986081 1239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:36.987229 1239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:36.987273 1239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:36.987300 1239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:36.988823 1239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab
--principal=oryx/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:36.989073 1239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:36.989308 1239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:36.994531 1251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:36.994537 1246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:36.994647 1239 server_base.cc:1034] running on GCE node
W20260805 22:42:36.994539 1247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:36.995033 1239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:36.995306 1239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:36.996496 1239 hybrid_clock.cc:648] HybridClock initialized: now 1785969756996480 us; error 32 us; skew 500 ppm
Aug 05 22:42:36 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:36.999217 1239 init.cc:377] Logged in from keytab as oryx/127.23.197.66@KRBTEST.COM (short username oryx)
I20260805 22:42:36.999809 1239 webserver.cc:460] Webserver started at http://127.23.197.66:35595/ using document root <none> and password file <none>
I20260805 22:42:37.000087 1239 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:37.000162 1239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:37.000334 1239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:37.001557 1239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "9c983822c3b841ff9f980aefcfb0a189"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "4c754bcf05f0baec0e0e693b1bcf2997"
server_key_iv: "2332df10333d79fa006c0de297dc8047"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.001962 1239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "9c983822c3b841ff9f980aefcfb0a189"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "4c754bcf05f0baec0e0e693b1bcf2997"
server_key_iv: "2332df10333d79fa006c0de297dc8047"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.003432 1239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:42:37.004246 1262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.004428 1239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:37.004545 1239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "9c983822c3b841ff9f980aefcfb0a189"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "4c754bcf05f0baec0e0e693b1bcf2997"
server_key_iv: "2332df10333d79fa006c0de297dc8047"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.004623 1239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:37.028533 1239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:37.030864 1239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:37.031059 1239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:37.031327 1239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:37.031665 1239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:37.031719 1239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.031762 1239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:37.031787 1239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.038034 1239 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:42627
I20260805 22:42:37.038136 1427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:42627 every 8 connection(s)
I20260805 22:42:37.038494 1239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:37.046948 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 1239
I20260805 22:42:37.047078 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:37.047286 24341 external_mini_cluster.cc:1428] Setting key 665f61e52fda90c62424431131e503bd
I20260805 22:42:37.048915 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.039469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:48437 (local address 127.23.197.126:37131)
0805 22:42:37.039563 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.039566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.039677 (+ 111us) server_negotiation.cc:407] Connection header received
0805 22:42:37.040217 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.040218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.040230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.040284 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.041351 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.041889 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.042945 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.043146 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.043218 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.045693 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.045694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.045694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.045702 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.047265 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.047766 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.047767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.047767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.047846 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.048157 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.048158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.048158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.048281 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.048338 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.048674 (+ 336us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.048844 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 22:42:37.049192 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.039391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.66:48437)
0805 22:42:37.039560 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.039600 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.040110 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.040352 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.040356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.041029 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.041250 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.041256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.041977 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.041979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.042837 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.042839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.043256 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.043257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.043373 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.044007 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.044022 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.045553 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.047349 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.047358 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.047361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.047680 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.047922 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.047923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.047924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.048087 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.048351 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.048358 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.048534 (+ 176us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.048652 (+ 118us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.048899 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260805 22:42:37.049705 1429 heartbeater.cc:347] Connected to a master server at 127.23.197.126:37131
I20260805 22:42:37.049849 1429 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:37.050103 1429 heartbeater.cc:511] Master 127.23.197.126:37131 requested a full tablet report, sending...
I20260805 22:42:37.050508 930 ts_manager.cc:194] Registered new tserver with Master: 9c983822c3b841ff9f980aefcfb0a189 (127.23.197.66:42627)
I20260805 22:42:37.050899 930 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:48437
WARNING: no policy specified for oryx/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab.
Entry for principal oryx/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab.
I20260805 22:42:37.075803 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:37131
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab
--principal=oryx/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:37.160614 1435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:37.160846 1435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:37.160871 1435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:37.162042 1435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:37.162086 1435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:37.162119 1435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:37.163703 1435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab
--principal=oryx/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:37.163961 1435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:37.164196 1435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:37.169729 1443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.169847 1440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.169718 1441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:37.170027 1435 server_base.cc:1034] running on GCE node
I20260805 22:42:37.170248 1435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:37.170471 1435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:37.171610 1435 hybrid_clock.cc:648] HybridClock initialized: now 1785969757171602 us; error 34 us; skew 500 ppm
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:37.174346 1435 init.cc:377] Logged in from keytab as oryx/127.23.197.67@KRBTEST.COM (short username oryx)
I20260805 22:42:37.174863 1435 webserver.cc:460] Webserver started at http://127.23.197.67:34539/ using document root <none> and password file <none>
I20260805 22:42:37.175109 1435 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:37.175177 1435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:37.175333 1435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:37.176427 1435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "a644a7911c594902955424ff55f8ac67"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "fa8216f4a433aade67a21047e77b470b"
server_key_iv: "34fedc74994a9f02adfde4f6be9394c9"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.176887 1435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "a644a7911c594902955424ff55f8ac67"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "fa8216f4a433aade67a21047e77b470b"
server_key_iv: "34fedc74994a9f02adfde4f6be9394c9"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.178295 1435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:37.179142 1450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.179342 1435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:37.179442 1435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "a644a7911c594902955424ff55f8ac67"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "fa8216f4a433aade67a21047e77b470b"
server_key_iv: "34fedc74994a9f02adfde4f6be9394c9"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.179522 1435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:37.190023 1435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:37.192049 1435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:37.192215 1435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:37.192446 1435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:37.192862 1435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:37.192914 1435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.192958 1435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:37.192987 1435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.199157 1435 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:45787
I20260805 22:42:37.199204 1618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:45787 every 8 connection(s)
I20260805 22:42:37.199544 1435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:37.201555 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 1435
I20260805 22:42:37.201642 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:37.201831 24341 external_mini_cluster.cc:1428] Setting key d0a83cde8e1980f44d883a6dcd516d21
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:37.209426 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.200426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:39259 (local address 127.23.197.126:37131)
0805 22:42:37.200489 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.200491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.200713 (+ 222us) server_negotiation.cc:407] Connection header received
0805 22:42:37.201336 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.201338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.201351 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.201403 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.202694 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.203330 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.204353 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.204543 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.204592 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.206739 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.206740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.206741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.206748 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.208037 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.208414 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.208415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.208415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.208467 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.208709 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.208710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.208710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.208941 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.209021 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.209272 (+ 251us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.209348 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 22:42:37.209570 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.200420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.67:39259)
0805 22:42:37.200592 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.200629 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.201195 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.201482 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.201485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.202310 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.202608 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.202615 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.203419 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.203420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.204263 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.204266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.204613 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.204614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.204716 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.205298 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.205308 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.206651 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.208109 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.208116 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.208118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.208349 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.208524 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.208525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.208525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.208626 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.209012 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.209017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.209138 (+ 121us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.209231 (+ 93us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.209354 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260805 22:42:37.210242 1619 heartbeater.cc:347] Connected to a master server at 127.23.197.126:37131
I20260805 22:42:37.210345 1619 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:37.210503 1619 heartbeater.cc:511] Master 127.23.197.126:37131 requested a full tablet report, sending...
I20260805 22:42:37.210850 930 ts_manager.cc:194] Registered new tserver with Master: a644a7911c594902955424ff55f8ac67 (127.23.197.67:45787)
I20260805 22:42:37.211187 930 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:39259
I20260805 22:42:37.213685 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.197.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.125.keytab.
Entry for principal oryx/127.23.197.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.125.keytab.
I20260805 22:42:37.244818 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.125:33535
--webserver_interface=127.23.197.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.125.keytab
--principal=oryx/127.23.197.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:37131,127.23.197.125:33535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:37.331548 1624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:37.331773 1624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:37.331797 1624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:37.333541 1624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:37.333586 1624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:37.333601 1624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:37.333614 1624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:37.333626 1624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:37.335246 1624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:37131,127.23.197.125:33535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.125.keytab
--principal=oryx/127.23.197.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.125:33535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.197.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
W20260805 22:42:37.335373 1624 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.23.197.126:37131,127.23.197.125:33535'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260805 22:42:37.335551 1624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:37.335844 1624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:37.341056 1633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.341125 1629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:37.341303 1624 server_base.cc:1034] running on GCE node
W20260805 22:42:37.341372 1631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:37.341627 1624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:37.341960 1624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:37.343133 1624 hybrid_clock.cc:648] HybridClock initialized: now 1785969757343105 us; error 48 us; skew 500 ppm
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:37.354873 1624 init.cc:377] Logged in from keytab as oryx/127.23.197.125@KRBTEST.COM (short username oryx)
I20260805 22:42:37.355443 1624 webserver.cc:460] Webserver started at http://127.23.197.125:43751/ using document root <none> and password file <none>
I20260805 22:42:37.355726 1624 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:37.355799 1624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:37.355955 1624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:37.357205 1624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data/instance:
uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "222ad6d19267823d39a9e8f21939a847"
server_key_iv: "352794f51c0794f515bea908d511431e"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.357609 1624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal/instance:
uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "222ad6d19267823d39a9e8f21939a847"
server_key_iv: "352794f51c0794f515bea908d511431e"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.359124 1624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:42:37.360070 1645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.360320 1624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:37.360442 1624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal
uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "222ad6d19267823d39a9e8f21939a847"
server_key_iv: "352794f51c0794f515bea908d511431e"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.360524 1624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:37.381165 1624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:37.383570 1624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:37.383734 1624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:37.388290 1624 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.125:33535
I20260805 22:42:37.388329 1723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.125:33535 every 8 connection(s)
I20260805 22:42:37.388710 1624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
I20260805 22:42:37.389796 1724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:37.391434 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 1624
I20260805 22:42:37.391584 1724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.125@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.197.125@KRBTEST.COM
I20260805 22:42:37.401670 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.392273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34170 (local address 127.23.197.126:37131)
0805 22:42:37.392327 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.392329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.392495 (+ 166us) server_negotiation.cc:407] Connection header received
0805 22:42:37.393084 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.393086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.393100 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.393185 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.393750 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.394269 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.395545 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.395812 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.395871 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.398377 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.398378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.398379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.398386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.399982 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.400423 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.400424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.400424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.400481 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.400826 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.400828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.400828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.401068 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.401166 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.401530 (+ 364us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.401603 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260805 22:42:37.401975 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.392237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34170)
0805 22:42:37.392371 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.392411 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.392922 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.393215 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.393217 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.393484 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.393645 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.393649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.394380 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.394381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.395435 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.395438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.395899 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.395900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.396021 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.396600 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.396609 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.398266 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.400060 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.400065 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.400067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.400349 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.400550 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.400551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.400552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.400690 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.401186 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.401192 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.401370 (+ 178us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.401497 (+ 127us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.401675 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":87,"threads_started":1}
I20260805 22:42:37.402549 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.392107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50728 (local address 127.23.197.125:33535)
0805 22:42:37.392323 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.392333 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.392367 (+ 34us) server_negotiation.cc:407] Connection header received
0805 22:42:37.392920 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.392929 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.392957 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.393060 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.393805 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.394845 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.395656 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.396079 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.396156 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.398758 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.398767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.398767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.398775 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.400791 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.401193 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.401196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.401196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.401267 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.401501 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.401502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.401502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.401718 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.402306 (+ 588us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.402355 (+ 49us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.402442 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
I20260805 22:42:37.402707 1724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.125@KRBTEST.COM for oryx/127.23.197.125@KRBTEST.COM
I20260805 22:42:37.409461 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.403016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.0.0.1:50742)
0805 22:42:37.403068 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.403078 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.403114 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.403266 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.403270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.403437 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.403485 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.403487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.404094 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.404095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.404575 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.404576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.404998 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.404999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.405060 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.405396 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.405400 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.406498 (+ 1098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.408374 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.408375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.408376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.408601 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.408886 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.408887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.408888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.408947 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.409177 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.409178 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.409218 (+ 40us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.409279 (+ 61us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.409361 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260805 22:42:37.409508 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.403094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50742 (local address 127.23.197.125:33535)
0805 22:42:37.403136 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.403138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.403148 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:37.403170 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.403171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.403180 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.403255 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.403554 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.404020 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.404652 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.404920 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.404999 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.406593 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.406594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.406595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.406601 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.408274 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.408691 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.408692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.408693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.408818 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.408999 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.408999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.408999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.409096 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.409143 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.409301 (+ 158us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.409408 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":22}
I20260805 22:42:37.410971 1724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: Bootstrap starting.
I20260805 22:42:37.411865 1724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:37.412129 1724 log.cc:824] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:37.412971 1724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: No bootstrap required, opened a new log
I20260805 22:42:37.414419 1724 raft_consensus.cc:348] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } }
I20260805 22:42:37.414554 1724 raft_consensus.cc:374] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:37.414578 1724 raft_consensus.cc:729] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c659c7f9c3be4fbab4c13e9de9e26bb8, State: Initialized, Role: FOLLOWER
I20260805 22:42:37.414669 1724 consensus_queue.cc:260] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } }
I20260805 22:42:37.414947 1731 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } } }
I20260805 22:42:37.414984 1724 sys_catalog.cc:564] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:37.415028 1731 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: This master's current role is: FOLLOWER
W20260805 22:42:37.417061 1748 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:42:37.417124 1748 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:42:37.419131 1624 master_runner.cc:428] Detected that this master c659c7f9c3be4fbab4c13e9de9e26bb8 is joining an existing cluster
I20260805 22:42:37.419273 1624 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.197.125:33535
I20260805 22:42:37.427148 1759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.420409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.0.0.1:50744)
0805 22:42:37.420596 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.420607 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.420664 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.420851 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.420852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.421030 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.421106 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.421108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.422055 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.422056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.422704 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.422707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.423201 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.423203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.423264 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.423748 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.423757 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.424115 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.426025 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.426026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.426026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.426273 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.426468 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.426469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.426470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.426526 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.426802 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.426802 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.426843 (+ 41us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.426908 (+ 65us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.427015 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":131,"threads_started":1}
I20260805 22:42:37.427155 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.420650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.23.197.125:33535)
0805 22:42:37.420688 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.420690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.420698 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:37.420740 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.420742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.420757 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.420849 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.421183 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.421962 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.422788 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.423079 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.423158 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.424202 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.424203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.424204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.424211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.425936 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.426343 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.426344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.426344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.426404 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.426594 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.426594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.426595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.426733 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.426811 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.426934 (+ 123us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.427076 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":20}
I20260805 22:42:37.431349 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.420655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.23.197.126:37131)
0805 22:42:37.420705 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.420706 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.424804 (+ 4098us) server_negotiation.cc:407] Connection header received
0805 22:42:37.424894 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.424895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.424908 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.424980 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.425564 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.426110 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.427228 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.427541 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.427592 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.428682 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.428684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.428684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.428691 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.430199 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.430585 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.430586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.430587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.430648 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.430838 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.430839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.430839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.430953 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.431022 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.431178 (+ 156us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.431258 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260805 22:42:37.431365 1760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.420423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34172)
0805 22:42:37.424704 (+ 4281us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.424715 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.424834 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.425201 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.425202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.425422 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.425478 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.425479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.426194 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.426195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.427134 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.427137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.427617 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.427618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.427688 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.428268 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.428273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.428571 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.430308 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.430309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.430310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.430496 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.430714 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.430715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.430715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.430780 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.431024 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.431025 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.431068 (+ 43us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.431148 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.431255 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4253,"thread_start_us":113,"threads_started":1}
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.126@KRBTEST.COM for oryx/127.23.197.125@KRBTEST.COM
I20260805 22:42:37.441248 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.434211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.23.197.125:33535)
0805 22:42:37.434249 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.434251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.434259 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:37.434334 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.434334 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.434344 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.434388 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.434835 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.435276 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.436052 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.436256 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.436348 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.438607 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.438608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.438609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.438616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.439983 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.440383 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.440384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.440385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.440443 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.440645 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.440646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.440646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.440753 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.440853 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.441056 (+ 203us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.441143 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260805 22:42:37.441262 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.433982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.0.0.1:50758)
0805 22:42:37.434135 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.434184 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.434263 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.434403 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.434407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.434692 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.434746 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.434748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.435349 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.435351 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.435965 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.435967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.436344 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.436345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.436440 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.436921 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.436926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.438507 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.440085 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.440094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.440096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.440306 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.440514 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.440515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.440515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.440579 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.440845 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.440853 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.440908 (+ 55us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.441021 (+ 113us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.441155 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260805 22:42:37.441936 930 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.197.125:33535: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8"
member_type: NON_VOTER
last_known_addr {
host: "127.23.197.125"
port: 33535
}
attrs {
promote: true
}
}
dest_uuid: "c0aa186bced34b6d9599b9126c511cbd"
cas_config_opid_index: -1
I20260805 22:42:37.442276 930 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } }
I20260805 22:42:37.442818 1764 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c659c7f9c3be4fbab4c13e9de9e26bb8 (127.23.197.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } } }
I20260805 22:42:37.443121 1681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:37.443903 1681 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Refusing update from remote peer c0aa186bced34b6d9599b9126c511cbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260805 22:42:37.443928 1764 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.197.125:33535
I20260805 22:42:37.444173 1763 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } } }
I20260805 22:42:37.444238 1624 master.cc:537] Master@127.23.197.125:33535 shutting down...
I20260805 22:42:37.444489 1764 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:37.445415 1763 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
W20260805 22:42:37.446463 910 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260805 22:42:37.446975 910 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd -> Peer c659c7f9c3be4fbab4c13e9de9e26bb8 (127.23.197.125:33535): Couldn't send request to peer c659c7f9c3be4fbab4c13e9de9e26bb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260805 22:42:37.447376 1624 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:37.447463 1624 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:37.447512 1624 tablet_replica.cc:330] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: stopping tablet replica
I20260805 22:42:37.461366 1624 master.cc:552] Master@127.23.197.125:33535 shutdown complete.
I20260805 22:42:37.461441 1624 master_runner.cc:305] Clearing existing system tablet
I20260805 22:42:37.461585 1624 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:37.461650 1624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:37.462435 1624 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:37.463004 1772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.463164 1624 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:37.463234 1624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:37.463253 1624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal
uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "222ad6d19267823d39a9e8f21939a847"
server_key_iv: "352794f51c0794f515bea908d511431e"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.464131 1624 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:37.465834 1624 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260805 22:42:37.465903 1624 log.cc:1196] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260805 22:42:37.466135 1624 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: Deleting consensus metadata
I20260805 22:42:37.466288 1624 master_runner.cc:315] Copying system tablet from 127.23.197.126:37131
I20260805 22:42:37.466933 1624 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: tablet copy: Beginning tablet copy session from remote peer at address 127.23.197.126:37131
I20260805 22:42:37.473853 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.467389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.23.197.126:37131)
0805 22:42:37.467443 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.467445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.467585 (+ 140us) server_negotiation.cc:407] Connection header received
0805 22:42:37.467672 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.467673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.467684 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.467745 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.468086 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.468600 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.469483 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.469698 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.469784 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.470802 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.470804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.470804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.470813 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.472528 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.472993 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.472995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.472995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.473078 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.473295 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.473296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.473296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.473425 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.473495 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.473657 (+ 162us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.473753 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260805 22:42:37.473870 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.467319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34176)
0805 22:42:37.467510 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.467520 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.467604 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.467750 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.467750 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.467940 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.468001 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.468002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.468694 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.468695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.469388 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.469391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.469788 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.469789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.469861 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.470369 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.470375 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.470704 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.472606 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.472607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.472608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.472924 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.473147 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.473148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.473149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.473224 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.473497 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.473498 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.473548 (+ 50us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.473630 (+ 82us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.473750 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260805 22:42:37.474471 959 tablet_copy_service.cc:140] P c0aa186bced34b6d9599b9126c511cbd: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c659c7f9c3be4fbab4c13e9de9e26bb8 ({username='oryx', principal='oryx/127.23.197.125@KRBTEST.COM'} at 127.0.0.1:34176)
I20260805 22:42:37.474566 959 tablet_copy_service.cc:161] P c0aa186bced34b6d9599b9126c511cbd: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c659c7f9c3be4fbab4c13e9de9e26bb8 at {username='oryx', principal='oryx/127.23.197.125@KRBTEST.COM'} at 127.0.0.1:34176: session id = c659c7f9c3be4fbab4c13e9de9e26bb8-00000000000000000000000000000000
I20260805 22:42:37.475247 959 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: Tablet Copy: opened 0 blocks and 1 log segments
I20260805 22:42:37.475939 1624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260805 22:42:37.477013 1624 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: tablet copy: Starting download of 0 data blocks...
I20260805 22:42:37.477200 1624 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: tablet copy: Starting download of 1 WAL segments...
I20260805 22:42:37.478075 1624 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260805 22:42:37.478946 959 tablet_copy_service.cc:342] P c0aa186bced34b6d9599b9126c511cbd: Request end of tablet copy session c659c7f9c3be4fbab4c13e9de9e26bb8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.197.125@KRBTEST.COM'} at 127.0.0.1:34176
I20260805 22:42:37.479007 959 tablet_copy_service.cc:433] P c0aa186bced34b6d9599b9126c511cbd: ending tablet copy session c659c7f9c3be4fbab4c13e9de9e26bb8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c659c7f9c3be4fbab4c13e9de9e26bb8
I20260805 22:42:37.481632 1624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:37.482923 1781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.483083 1784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.483219 1782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:37.483296 1624 server_base.cc:1034] running on GCE node
I20260805 22:42:37.483393 1624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:37.483518 1624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:37.484624 1624 hybrid_clock.cc:648] HybridClock initialized: now 1785969757484603 us; error 38 us; skew 500 ppm
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:37.486130 1624 init.cc:377] Logged in from keytab as oryx/127.23.197.125@KRBTEST.COM (short username oryx)
I20260805 22:42:37.486429 1624 webserver.cc:460] Webserver started at http://127.23.197.125:40425/ using document root <none> and password file <none>
I20260805 22:42:37.486565 1624 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:37.486611 1624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:37.487263 1624 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:37.487835 1792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.488003 1624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:37.488060 1624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal
uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "222ad6d19267823d39a9e8f21939a847"
server_key_iv: "352794f51c0794f515bea908d511431e"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.488124 1624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:37.502913 1624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:37.504895 1624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:37.508216 1624 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.125:33535
I20260805 22:42:37.508244 1863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.125:33535 every 8 connection(s)
I20260805 22:42:37.508589 1624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
I20260805 22:42:37.509442 1864 sys_catalog.cc:263] Verifying existing consensus state
I20260805 22:42:37.509893 1864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: Bootstrap starting.
I20260805 22:42:37.512161 1864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:42:37.512398 1864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: Bootstrap complete.
I20260805 22:42:37.512607 1864 raft_consensus.cc:348] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } }
I20260805 22:42:37.512679 1864 raft_consensus.cc:729] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c659c7f9c3be4fbab4c13e9de9e26bb8, State: Initialized, Role: LEARNER
I20260805 22:42:37.512707 1864 consensus_queue.cc:260] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } }
I20260805 22:42:37.512945 1864 sys_catalog.cc:564] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:37.514190 1867 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } } }
I20260805 22:42:37.514401 1881 catalog_manager.cc:1259] Loaded cluster ID: 989888fdf71e43e4b1459dcdf7f54a1b
I20260805 22:42:37.514482 1867 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: This master's current role is: LEARNER
I20260805 22:42:37.514683 1881 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: loading cluster ID for follower catalog manager: success
I20260805 22:42:37.515705 1881 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: acquiring CA information for follower catalog manager: success
I20260805 22:42:37.515995 1881 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 22:42:37.562705 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.555041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.23.197.125:33535)
0805 22:42:37.555235 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.555237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.555248 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:37.555277 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.555278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.555290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.555377 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.555930 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.557095 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.557881 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.558262 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.558358 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.559632 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.559633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.559633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.559641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.561400 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.561783 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.561785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.561786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.561871 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.562070 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.562071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.562071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.562232 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.562338 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.562490 (+ 152us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.562586 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for oryx/127.23.197.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.124.keytab.
Entry for principal oryx/127.23.197.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.124.keytab.
I20260805 22:42:37.591822 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.124:40965
--webserver_interface=127.23.197.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.124.keytab
--principal=oryx/127.23.197.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:37131,127.23.197.125:33535,127.23.197.124:40965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:37.683102 1888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:37.683350 1888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:37.683374 1888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:37.684551 1888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:37.684598 1888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:37.684614 1888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:37.684628 1888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:37.684639 1888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:37.686187 1888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:37131,127.23.197.125:33535,127.23.197.124:40965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.124.keytab
--principal=oryx/127.23.197.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.124:40965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.197.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:37.686410 1888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:37.686606 1888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:37.691937 1896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.691944 1901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.691963 1897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:37.692499 1888 server_base.cc:1034] running on GCE node
I20260805 22:42:37.692824 1888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:37.693145 1888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:37.694341 1888 hybrid_clock.cc:648] HybridClock initialized: now 1785969757694327 us; error 32 us; skew 500 ppm
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:37.697163 1888 init.cc:377] Logged in from keytab as oryx/127.23.197.124@KRBTEST.COM (short username oryx)
I20260805 22:42:37.697592 1888 webserver.cc:460] Webserver started at http://127.23.197.124:40067/ using document root <none> and password file <none>
I20260805 22:42:37.697845 1888 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:37.697916 1888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:37.698060 1888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:37.699168 1888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data/instance:
uuid: "3cd5e2e334ad4ed296b517a9335594f6"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "737960321231c4c321d37fcf0f663130"
server_key_iv: "d079ec88d42a3051ee415ab76030fbea"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.699541 1888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal/instance:
uuid: "3cd5e2e334ad4ed296b517a9335594f6"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "737960321231c4c321d37fcf0f663130"
server_key_iv: "d079ec88d42a3051ee415ab76030fbea"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.701088 1888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260805 22:42:37.701970 1914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.702170 1888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:37.702301 1888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal
uuid: "3cd5e2e334ad4ed296b517a9335594f6"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "737960321231c4c321d37fcf0f663130"
server_key_iv: "d079ec88d42a3051ee415ab76030fbea"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.702378 1888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:37.714708 1888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:37.717042 1888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:37.717219 1888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:37.721717 1888 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.124:40965
I20260805 22:42:37.721756 1989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.124:40965 every 8 connection(s)
I20260805 22:42:37.722141 1888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
I20260805 22:42:37.723244 1990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:37.724972 1990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:42:37.727907 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 1888
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.124@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.197.124@KRBTEST.COM
I20260805 22:42:37.735443 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.725839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.23.197.126:37131)
0805 22:42:37.725934 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.725936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.726073 (+ 137us) server_negotiation.cc:407] Connection header received
0805 22:42:37.726533 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.726534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.726547 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.726607 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.727259 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.727769 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.729259 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.729439 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.729513 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.732331 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.732333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.732333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.732345 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.733849 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.734259 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.734260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.734261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.734323 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.734618 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.734619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.734619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.734813 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.734899 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.735228 (+ 329us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.735321 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 22:42:37.735658 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.725714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34188)
0805 22:42:37.725944 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.725986 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.726441 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.726684 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.726699 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.726969 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.727171 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.727175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.727862 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.727864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.729169 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.729172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.729544 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.729545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.729664 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.730220 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.730229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.732206 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.733933 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.733941 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.733943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.734180 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.734390 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.734391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.734392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.734537 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.734905 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.734911 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.735055 (+ 144us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.735173 (+ 118us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.735406 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":127,"threads_started":1}
I20260805 22:42:37.736383 1990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:42:37.737774 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.728473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.23.197.124:40965)
0805 22:42:37.728647 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.728651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.728666 (+ 15us) server_negotiation.cc:407] Connection header received
0805 22:42:37.728684 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.728685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.728701 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.728830 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.729541 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.730244 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.731199 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.731692 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.731764 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.734298 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.734307 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.734308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.734315 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.736230 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.736629 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.736632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.736633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.736700 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.737076 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.737077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.737077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.737283 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.737431 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.737523 (+ 92us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.737640 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.124@KRBTEST.COM for oryx/127.23.197.125@KRBTEST.COM
I20260805 22:42:37.743578 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.736695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.0.0.1:50792)
0805 22:42:37.736869 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.736899 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.736967 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.737134 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.737135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.737255 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.737306 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.737307 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.738221 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.738222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.738816 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.738818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.739198 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.739199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.739270 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.739829 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.739835 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.741059 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.742751 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.742752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.742753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.742917 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.743088 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.743089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.743089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.743140 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.743344 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.743345 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.743382 (+ 37us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.743436 (+ 54us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.743508 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260805 22:42:37.743578 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.736873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.23.197.125:33535)
0805 22:42:37.736926 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.736928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.736940 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:42:37.737038 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.737039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.737062 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.737126 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.737384 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.738140 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.738902 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.739128 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.739203 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.741143 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.741144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.741144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.741151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.742669 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.742981 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.742982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.742982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.743032 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.743190 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.743191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.743191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.743292 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.743342 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.743451 (+ 109us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.743519 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260805 22:42:37.744026 1990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.124@KRBTEST.COM for oryx/127.23.197.124@KRBTEST.COM
I20260805 22:42:37.751019 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.744287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.0.0.1:40180)
0805 22:42:37.744335 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.744347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.744386 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.744497 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.744498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.744612 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.744642 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.744643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.745310 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.745311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.745803 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.745804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.746159 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.746160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.746205 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.746529 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.746533 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.747941 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.749929 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.749930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.749931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.750144 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.750359 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.750360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.750360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.750417 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.750675 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.750676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.750719 (+ 43us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.750797 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.750900 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260805 22:42:37.751030 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.744337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40180 (local address 127.23.197.124:40965)
0805 22:42:37.744384 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.744386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.744395 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:37.744442 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.744443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.744452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.744495 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.744692 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.745232 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.745895 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.746089 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.746139 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.748026 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.748027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.748028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.748035 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.749823 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.750237 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.750238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.750238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.750290 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.750482 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.750482 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.750483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.750601 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.750673 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.750822 (+ 149us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.750951 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":29}
I20260805 22:42:37.752650 1990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Bootstrap starting.
I20260805 22:42:37.753611 1990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:37.753968 1990 log.cc:824] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:37.754760 1990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: No bootstrap required, opened a new log
I20260805 22:42:37.756435 1990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } }
I20260805 22:42:37.756567 1990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:37.756592 1990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd5e2e334ad4ed296b517a9335594f6, State: Initialized, Role: FOLLOWER
I20260805 22:42:37.756728 1990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } }
I20260805 22:42:37.757087 1998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } } }
I20260805 22:42:37.757153 1990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:37.757205 1998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: This master's current role is: FOLLOWER
W20260805 22:42:37.758945 2011 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:42:37.759006 2011 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:42:37.761611 1888 master_runner.cc:428] Detected that this master 3cd5e2e334ad4ed296b517a9335594f6 is joining an existing cluster
I20260805 22:42:37.761677 1888 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.197.124:40965
I20260805 22:42:37.770031 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.763302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.23.197.124:40965)
0805 22:42:37.763344 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.763347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.763678 (+ 331us) server_negotiation.cc:407] Connection header received
0805 22:42:37.763801 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.763802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.763816 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.763867 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.764279 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.765035 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.765852 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.766135 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.766186 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.767339 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.767340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.767341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.767347 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.768837 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.769199 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.769200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.769200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.769254 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.769448 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.769448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.769449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.769562 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.769636 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.769809 (+ 173us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.769915 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":21}
I20260805 22:42:37.770128 2018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.763089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.0.0.1:40190)
0805 22:42:37.763608 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.763618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.763708 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.763917 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.763918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.764112 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.764197 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.764199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.765125 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.765126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.765742 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.765744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.766266 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.766266 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.766330 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.766864 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.766870 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.767253 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.768937 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.768939 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.768939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.769116 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.769327 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.769328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.769328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.769390 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.769651 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.769652 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.769714 (+ 62us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.769786 (+ 72us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.769985 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":97,"thread_start_us":74,"threads_started":1}
I20260805 22:42:37.789517 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 1023
I20260805 22:42:37.791329 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.763288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.23.197.126:37131)
0805 22:42:37.763338 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.763340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.763484 (+ 144us) server_negotiation.cc:407] Connection header received
0805 22:42:37.763561 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.763563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.763576 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.763650 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.764203 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.765055 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.766146 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.766479 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.766545 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.787276 (+ 20731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.787278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.787278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.787295 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.789290 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.790231 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.790238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.790238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.790312 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.790524 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.790525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.790525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.790693 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.790746 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.791057 (+ 311us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.791168 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:42:37.791841 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.763033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34204)
0805 22:42:37.763402 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.763412 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.763496 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.763670 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.763671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.764002 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.764093 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.764096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.765150 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.765151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.766057 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.766060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.766613 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.766614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.766687 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.786686 (+ 19999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.786693 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.787174 (+ 481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.789417 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.789418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.789419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.789662 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.790389 (+ 727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.790390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.790391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.790456 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.790807 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.790808 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.790894 (+ 86us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.791492 (+ 598us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.791661 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":124,"threads_started":1}
I20260805 22:42:37.796195 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.763110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50798 (local address 127.23.197.125:33535)
0805 22:42:37.763164 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.763182 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.763885 (+ 703us) server_negotiation.cc:407] Connection header received
0805 22:42:37.766650 (+ 2765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.766651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.766666 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.766699 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.785440 (+ 18741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.785967 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.787139 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.787469 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.787562 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.788745 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.788747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.788747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.788756 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.795014 (+ 6258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.795405 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.795406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.795406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.795458 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.795622 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.795623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.795623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.795745 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.795783 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.795949 (+ 166us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.796031 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260805 22:42:37.796819 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.763227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.0.0.1:50798)
0805 22:42:37.763827 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.763836 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.766518 (+ 2682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.766789 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.766789 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.785201 (+ 18412us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.785300 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.785306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.786072 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.786074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.787005 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.787009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.787597 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.787599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.787675 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.788088 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.788094 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.788620 (+ 526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.795120 (+ 6500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.795121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.795121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.795333 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.795517 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.795517 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.795517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.795567 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.795820 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.795821 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.795860 (+ 39us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.796522 (+ 662us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.796646 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":87,"threads_started":1}
I20260805 22:42:37.811245 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:33123
--local_ip_for_outbound_sockets=127.23.197.65
--tserver_master_addrs=127.23.197.126:37131
--webserver_port=36663
--webserver_interface=127.23.197.65
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab
--principal=oryx/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.197.126:37131,127.23.197.125:33535
--tserver_master_addrs=127.23.197.126:37131,127.23.197.125:33535,127.23.197.124:40965 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969756, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.126@KRBTEST.COM for oryx/127.23.197.124@KRBTEST.COM
I20260805 22:42:37.816045 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.806963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.0.0.1:40202)
0805 22:42:37.807019 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.807030 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.807072 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.807360 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.807361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.807591 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.807652 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.807654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.808660 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.808662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.809585 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.809589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.810082 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.810083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.810202 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.810730 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.810736 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.812465 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.814747 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.814749 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.814750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.815016 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.815275 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.815277 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.815277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.815356 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.815673 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.815674 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.815731 (+ 57us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.815809 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.815937 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260805 22:42:37.816087 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.807017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40202 (local address 127.23.197.124:40965)
0805 22:42:37.807165 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.807167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.807175 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:37.807234 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.807235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.807246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.807293 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.807739 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.808554 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.809673 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.809997 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.810071 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.812578 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.812580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.812580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.812588 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.814609 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.815101 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.815102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.815103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.815179 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.815428 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.815429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.815430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.815595 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.815673 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.815850 (+ 177us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.815946 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260805 22:42:37.816700 930 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.197.124:40965: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6"
member_type: NON_VOTER
last_known_addr {
host: "127.23.197.124"
port: 40965
}
attrs {
promote: true
}
}
dest_uuid: "c0aa186bced34b6d9599b9126c511cbd"
cas_config_opid_index: 5
I20260805 22:42:37.816999 930 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } }
I20260805 22:42:37.817507 1763 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3cd5e2e334ad4ed296b517a9335594f6 (127.23.197.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.818624 1763 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.197.124:40965
I20260805 22:42:37.818706 1763 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.818779 1763 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:37.819058 1888 master.cc:537] Master@127.23.197.124:40965 shutting down...
I20260805 22:42:37.819331 1950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:37.820374 1950 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 FOLLOWER]: Refusing update from remote peer c0aa186bced34b6d9599b9126c511cbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260805 22:42:37.820658 1764 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260805 22:42:37.821534 906 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd -> Peer 3cd5e2e334ad4ed296b517a9335594f6 (127.23.197.124:40965): Couldn't send request to peer 3cd5e2e334ad4ed296b517a9335594f6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260805 22:42:37.823864 1888 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:37.823971 1888 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:37.824028 1888 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: stopping tablet replica
I20260805 22:42:37.825723 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.817456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.0.0.1:50814)
0805 22:42:37.821665 (+ 4209us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.821676 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.821719 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.821946 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.821947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.821949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:37.821997 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.821998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.823028 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.823033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.824256 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.824260 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.825457 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.825458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.825519 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.825547 (+ 28us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.825600 (+ 53us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.825646 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4193}
I20260805 22:42:37.825922 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.819012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50814 (local address 127.23.197.125:33535)
0805 22:42:37.821631 (+ 2619us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.821634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.821769 (+ 135us) server_negotiation.cc:407] Connection header received
0805 22:42:37.821796 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.821797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.821824 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.821879 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:37.822065 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.822821 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.824370 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.825261 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.825337 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.825422 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.825821 (+ 399us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.825867 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2588}
I20260805 22:42:37.826169 1828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 LEARNER]: Refusing update from remote peer c0aa186bced34b6d9599b9126c511cbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260805 22:42:37.826426 1764 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260805 22:42:37.827157 1828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3cd5e2e334ad4ed296b517a9335594f6 (127.23.197.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.828549 1764 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: attempting to promote NON_VOTER c659c7f9c3be4fbab4c13e9de9e26bb8 to VOTER
I20260805 22:42:37.828605 1867 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0aa186bced34b6d9599b9126c511cbd. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.828691 1867 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: This master's current role is: LEARNER
I20260805 22:42:37.828727 1867 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: NON_VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: true } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.828753 1764 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } }
I20260805 22:42:37.828871 1867 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: This master's current role is: LEARNER
I20260805 22:42:37.829361 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.829217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.0.0.1:40214)
0805 22:42:37.829278 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.829331 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:40965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260805 22:42:37.829480 1828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 LEARNER]: Refusing update from remote peer c0aa186bced34b6d9599b9126c511cbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260805 22:42:37.829653 1764 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260805 22:42:37.829838 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.829709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.0.0.1:40228)
0805 22:42:37.829790 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.829814 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:40965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260805 22:42:37.829960 906 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd -> Peer 3cd5e2e334ad4ed296b517a9335594f6 (127.23.197.124:40965): Couldn't send request to peer 3cd5e2e334ad4ed296b517a9335594f6. Status: Network error: Client connection negotiation failed: client connection to 127.23.197.124:40965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260805 22:42:37.830256 1764 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c659c7f9c3be4fbab4c13e9de9e26bb8 (127.23.197.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.831211 1764 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.831233 1763 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.831348 1763 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:37.831313 1764 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:37.831560 1828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c659c7f9c3be4fbab4c13e9de9e26bb8 (127.23.197.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.832425 2025 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.832574 2025 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:42:37.838668 1888 master.cc:552] Master@127.23.197.124:40965 shutdown complete.
I20260805 22:42:37.838742 1888 master_runner.cc:305] Clearing existing system tablet
I20260805 22:42:37.838879 1888 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:37.838935 1888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:37.839752 1888 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260805 22:42:37.840384 2028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.840647 1888 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:37.840734 1888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:42:37.840792 1888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal
uuid: "3cd5e2e334ad4ed296b517a9335594f6"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "737960321231c4c321d37fcf0f663130"
server_key_iv: "d079ec88d42a3051ee415ab76030fbea"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.841638 1888 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:37.843580 1888 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260805 22:42:37.843690 1888 log.cc:1196] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260805 22:42:37.843915 1888 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Deleting consensus metadata
I20260805 22:42:37.844045 1888 master_runner.cc:315] Copying system tablet from 127.23.197.126:37131
I20260805 22:42:37.844709 1888 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: tablet copy: Beginning tablet copy session from remote peer at address 127.23.197.126:37131
I20260805 22:42:37.856328 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.845157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34216 (local address 127.23.197.126:37131)
0805 22:42:37.845210 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.845212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.849060 (+ 3848us) server_negotiation.cc:407] Connection header received
0805 22:42:37.849134 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.849136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.849153 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.849232 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.849663 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.850455 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.851542 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.851831 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.851908 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.853175 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.853177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.853177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.853186 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.854958 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.855435 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.855437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.855437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.855506 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.855740 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.855741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.855741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.855879 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.855949 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.856146 (+ 197us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.856232 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260805 22:42:37.856343 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.845067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34216)
0805 22:42:37.848937 (+ 3870us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.848956 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.849063 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.849253 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.849254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.849496 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.849556 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.849560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.850563 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.850565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.851440 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.851443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.851931 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.851933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.851998 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.852565 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.852570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.853076 (+ 506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.855089 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.855091 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.855092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.855350 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.855585 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.855586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.855586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.855667 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.855957 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.855958 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.856015 (+ 57us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.856099 (+ 84us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.856233 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3836,"thread_start_us":86,"threads_started":1}
I20260805 22:42:37.857026 959 tablet_copy_service.cc:140] P c0aa186bced34b6d9599b9126c511cbd: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3cd5e2e334ad4ed296b517a9335594f6 ({username='oryx', principal='oryx/127.23.197.124@KRBTEST.COM'} at 127.0.0.1:34216)
I20260805 22:42:37.857092 959 tablet_copy_service.cc:161] P c0aa186bced34b6d9599b9126c511cbd: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3cd5e2e334ad4ed296b517a9335594f6 at {username='oryx', principal='oryx/127.23.197.124@KRBTEST.COM'} at 127.0.0.1:34216: session id = 3cd5e2e334ad4ed296b517a9335594f6-00000000000000000000000000000000
I20260805 22:42:37.857700 959 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: Tablet Copy: opened 0 blocks and 1 log segments
I20260805 22:42:37.858367 1888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260805 22:42:37.859843 1888 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: tablet copy: Starting download of 0 data blocks...
I20260805 22:42:37.859989 1888 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: tablet copy: Starting download of 1 WAL segments...
I20260805 22:42:37.860966 1888 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260805 22:42:37.862138 959 tablet_copy_service.cc:342] P c0aa186bced34b6d9599b9126c511cbd: Request end of tablet copy session 3cd5e2e334ad4ed296b517a9335594f6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.197.124@KRBTEST.COM'} at 127.0.0.1:34216
I20260805 22:42:37.862201 959 tablet_copy_service.cc:433] P c0aa186bced34b6d9599b9126c511cbd: ending tablet copy session 3cd5e2e334ad4ed296b517a9335594f6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3cd5e2e334ad4ed296b517a9335594f6
I20260805 22:42:37.865327 1888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:37.866564 2043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.866780 2040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.866964 2041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:37.868976 1888 server_base.cc:1034] running on GCE node
I20260805 22:42:37.869109 1888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:37.869261 1888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:37.870389 1888 hybrid_clock.cc:648] HybridClock initialized: now 1785969757870368 us; error 40 us; skew 500 ppm
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:37.872021 1888 init.cc:377] Logged in from keytab as oryx/127.23.197.124@KRBTEST.COM (short username oryx)
I20260805 22:42:37.872328 1888 webserver.cc:460] Webserver started at http://127.23.197.124:35859/ using document root <none> and password file <none>
I20260805 22:42:37.872457 1888 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:37.872499 1888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:37.873366 1888 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:42:37.874024 2051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.874214 1888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:37.874275 1888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal
uuid: "3cd5e2e334ad4ed296b517a9335594f6"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "737960321231c4c321d37fcf0f663130"
server_key_iv: "d079ec88d42a3051ee415ab76030fbea"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.874343 1888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:37.889520 1888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:37.891505 1888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:37.894819 1888 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.124:40965
I20260805 22:42:37.895198 1888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
I20260805 22:42:37.896214 2127 sys_catalog.cc:263] Verifying existing consensus state
I20260805 22:42:37.896747 2127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Bootstrap starting.
I20260805 22:42:37.896956 2126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.124:40965 every 8 connection(s)
I20260805 22:42:37.900015 2127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:42:37.900265 2127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Bootstrap complete.
I20260805 22:42:37.900487 2127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } }
I20260805 22:42:37.900584 2127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3cd5e2e334ad4ed296b517a9335594f6, State: Initialized, Role: LEARNER
I20260805 22:42:37.900624 2127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } }
I20260805 22:42:37.900868 2127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:37.900856 2131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:37.901110 2131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: This master's current role is: LEARNER
I20260805 22:42:37.902472 2147 catalog_manager.cc:1259] Loaded cluster ID: 989888fdf71e43e4b1459dcdf7f54a1b
I20260805 22:42:37.902568 2147 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: loading cluster ID for follower catalog manager: success
I20260805 22:42:37.903909 2147 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: acquiring CA information for follower catalog manager: success
I20260805 22:42:37.904244 2147 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260805 22:42:37.920481 2022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:37.920720 2022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:37.920747 2022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:37.922089 2022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:37.922144 2022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:37.922175 2022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:37.924060 2022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab
--principal=oryx/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:33123
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=36663
--tserver_master_addrs=127.23.197.126:37131,127.23.197.125:33535,127.23.197.124:40965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:37.924343 2022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:37.924602 2022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:37.930707 2156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.930750 2159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:37.930847 2154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:37.931067 2022 server_base.cc:1034] running on GCE node
I20260805 22:42:37.931339 2022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:37.931625 2022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:37.932806 2022 hybrid_clock.cc:648] HybridClock initialized: now 1785969757932785 us; error 38 us; skew 500 ppm
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:37.936069 2022 init.cc:377] Logged in from keytab as oryx/127.23.197.65@KRBTEST.COM (short username oryx)
I20260805 22:42:37.936595 2022 webserver.cc:460] Webserver started at http://127.23.197.65:36663/ using document root <none> and password file <none>
I20260805 22:42:37.936956 2022 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:37.937057 2022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:37.938601 2022 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:37.939414 2166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.939666 2022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:37.939776 2022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "0437f45cd9544ee0aacaeadd7a5b0f34"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "d736fd692ccc5aa389c312d5b647d501"
server_key_iv: "fd43a4f70a6ecd4061a60d0388a614cc"
server_key_version: "encryptionkey@0"
I20260805 22:42:37.940150 2022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:37.963083 2022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:37.965762 2022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:37.965984 2022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:37.966256 2022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:37.966720 2022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:37.966773 2022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.966825 2022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:37.966853 2022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:37.973323 2022 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:33123
I20260805 22:42:37.973373 2329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:33123 every 8 connection(s)
I20260805 22:42:37.973753 2022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:37.979911 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 2022
I20260805 22:42:37.980113 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 1239
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.125@KRBTEST.COM
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.124@KRBTEST.COM
I20260805 22:42:37.986814 2336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.974632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.23.197.65:58319)
0805 22:42:37.975029 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.975039 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.975567 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.975876 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.975880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.976263 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.976483 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.976487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.977529 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.977530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.978415 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.978418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.978845 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.978846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.978981 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.979530 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.979538 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.981899 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.985103 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.985114 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.985117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.985447 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.985637 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.985637 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.985638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.985775 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.986065 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.986070 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.986235 (+ 165us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.986331 (+ 96us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.986516 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260805 22:42:37.987172 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.974949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:58319 (local address 127.23.197.125:33535)
0805 22:42:37.975039 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.975041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.975138 (+ 97us) server_negotiation.cc:407] Connection header received
0805 22:42:37.975725 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.975727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.975741 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.975803 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.976580 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.977428 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.978502 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.978764 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.978831 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.982003 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.982005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.982005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.982013 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.985004 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.985523 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.985524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.985525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.985580 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.985841 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.985842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.985842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.985989 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.986043 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.986958 (+ 915us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.987045 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 22:42:37.990724 2332 heartbeater.cc:347] Connected to a master server at 127.23.197.125:33535
I20260805 22:42:37.990844 2332 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:37.991073 2332 heartbeater.cc:511] Master 127.23.197.125:33535 requested a full tablet report, sending...
I20260805 22:42:37.991742 1813 ts_manager.cc:194] Registered new tserver with Master: 0437f45cd9544ee0aacaeadd7a5b0f34 (127.23.197.65:33123)
I20260805 22:42:37.994701 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.974632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.23.197.65:34951)
0805 22:42:37.974859 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.974897 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.975565 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.976016 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.976017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.976304 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.976483 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.976489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.977471 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.977472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.978626 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.978628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.979133 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.979134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.979219 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.979874 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.979879 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.991339 (+ 11460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.993496 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.993498 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.993498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.993712 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.993950 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.993951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.993951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.994009 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.994308 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.994310 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.994364 (+ 54us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.994455 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.994576 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":82,"thread_start_us":140,"threads_started":1}
Aug 05 22:42:37 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969757, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:37.995368 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.975024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:34951 (local address 127.23.197.124:40965)
0805 22:42:37.975216 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.975219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.975227 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:37.975864 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.975865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.975879 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.975966 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.976603 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.977362 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.978715 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.979009 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.979075 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.991452 (+ 12377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.991454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.991454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.991468 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.993379 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.993806 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.993807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.993808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.993878 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.994078 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.994079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.994080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.994230 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.994314 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.995217 (+ 903us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.995300 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":61,"threads_started":1}
I20260805 22:42:37.996012 2330 heartbeater.cc:347] Connected to a master server at 127.23.197.124:40965
I20260805 22:42:37.996093 2330 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:37.996155 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:42627
--local_ip_for_outbound_sockets=127.23.197.66
--tserver_master_addrs=127.23.197.126:37131
--webserver_port=35595
--webserver_interface=127.23.197.66
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab
--principal=oryx/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.197.126:37131,127.23.197.125:33535
--tserver_master_addrs=127.23.197.126:37131,127.23.197.125:33535,127.23.197.124:40965 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:42:37.996196 2330 heartbeater.cc:511] Master 127.23.197.124:40965 requested a full tablet report, sending...
I20260805 22:42:37.996800 2076 ts_manager.cc:194] Registered new tserver with Master: 0437f45cd9544ee0aacaeadd7a5b0f34 (127.23.197.65:33123)
I20260805 22:42:37.998850 2335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.974633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.65:47419)
0805 22:42:37.975203 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:37.975215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:37.975581 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:37.976333 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:37.976333 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:37.976477 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:37.976538 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.976540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.977495 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.977496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.978597 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:37.978601 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.979035 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.979036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:37.979103 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.979675 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:37.979680 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:37.995426 (+ 15746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:37.997592 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.997593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.997593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.997910 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.998167 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:37.998168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:37.998168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:37.998223 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:37.998538 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:37.998538 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:37.998574 (+ 36us) client_negotiation.cc:769] Sending connection context
0805 22:42:37.998626 (+ 52us) client_negotiation.cc:240] Negotiation successful
0805 22:42:37.998714 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":338,"thread_start_us":84,"threads_started":1}
I20260805 22:42:37.998884 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:37.975097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:47419 (local address 127.23.197.126:37131)
0805 22:42:37.975166 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:37.975169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:37.975269 (+ 100us) server_negotiation.cc:407] Connection header received
0805 22:42:37.975725 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:37.975727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:37.975741 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:37.975776 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:37.976666 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.977433 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.978690 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:37.978960 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:37.979034 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:37.995545 (+ 16511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:37.995547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:37.995547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:37.995557 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:37.997502 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.998004 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.998005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.998006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.998082 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:37.998288 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:37.998289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:37.998289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:37.998469 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:37.998552 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:37.998658 (+ 106us) server_negotiation.cc:299] Negotiation successful
0805 22:42:37.998758 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260805 22:42:37.999490 2331 heartbeater.cc:347] Connected to a master server at 127.23.197.126:37131
I20260805 22:42:37.999544 2331 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:37.999632 2331 heartbeater.cc:511] Master 127.23.197.126:37131 requested a full tablet report, sending...
I20260805 22:42:37.999985 930 ts_manager.cc:194] Re-registered known tserver with Master: 0437f45cd9544ee0aacaeadd7a5b0f34 (127.23.197.65:33123)
I20260805 22:42:38.000349 930 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:47419
W20260805 22:42:38.081694 2339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:38.082033 2339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:38.082103 2339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:38.083361 2339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:38.083418 2339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:38.083472 2339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:38.085028 2339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab
--principal=oryx/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:42627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=35595
--tserver_master_addrs=127.23.197.126:37131,127.23.197.125:33535,127.23.197.124:40965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:38.085309 2339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:38.085548 2339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:38.091302 2355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:38.091413 2350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:38.091506 2346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:38.091951 2339 server_base.cc:1034] running on GCE node
I20260805 22:42:38.092173 2339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:38.092427 2339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:38.093566 2339 hybrid_clock.cc:648] HybridClock initialized: now 1785969758093547 us; error 38 us; skew 500 ppm
Aug 05 22:42:38 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969758, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:38.096153 2339 init.cc:377] Logged in from keytab as oryx/127.23.197.66@KRBTEST.COM (short username oryx)
I20260805 22:42:38.096619 2339 webserver.cc:460] Webserver started at http://127.23.197.66:35595/ using document root <none> and password file <none>
I20260805 22:42:38.096885 2339 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:38.096962 2339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:38.098417 2339 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:38.099080 2368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:38.099255 2339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:38.099359 2339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "9c983822c3b841ff9f980aefcfb0a189"
format_stamp: "Formatted at 2026-08-05 22:42:36 on dist-test-slave-g170"
server_key: "4c754bcf05f0baec0e0e693b1bcf2997"
server_key_iv: "2332df10333d79fa006c0de297dc8047"
server_key_version: "encryptionkey@0"
I20260805 22:42:38.099714 2339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:38.112068 2339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:38.114558 2339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:38.114717 2339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:38.114977 2339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:38.115307 2339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:38.115358 2339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:38.115393 2339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:38.115419 2339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:38.123236 2339 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:42627
I20260805 22:42:38.123281 2533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:42627 every 8 connection(s)
I20260805 22:42:38.123660 2339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
Aug 05 22:42:38 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969758, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
Aug 05 22:42:38 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969758, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.125@KRBTEST.COM
Aug 05 22:42:38 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969758, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.124@KRBTEST.COM
I20260805 22:42:38.133167 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 2339
I20260805 22:42:38.133350 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 1435
I20260805 22:42:38.138635 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.124525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:36959 (local address 127.23.197.126:37131)
0805 22:42:38.124590 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.124592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.125012 (+ 420us) server_negotiation.cc:407] Connection header received
0805 22:42:38.126050 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.126051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.126065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.126099 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.126908 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.127427 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.128870 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.129181 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.129264 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.133124 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.133125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.133125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.133133 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.136289 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.137277 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.137278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.137279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.137348 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.137673 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.137675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.137675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.137845 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.137906 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.138359 (+ 453us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.138464 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260805 22:42:38.138661 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.124748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:45849 (local address 127.23.197.124:40965)
0805 22:42:38.124877 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.124880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.125003 (+ 123us) server_negotiation.cc:407] Connection header received
0805 22:42:38.126026 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.126028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.126040 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.126090 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.126885 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.127703 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.130091 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.130413 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.130495 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.133123 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.133125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.133125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.133133 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.137359 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.137782 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.137783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.137784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.137841 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.138050 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.138051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.138051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.138192 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.138246 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.138457 (+ 211us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.138561 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260805 22:42:38.138761 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.124609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:44617 (local address 127.23.197.125:33535)
0805 22:42:38.124656 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.124658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.124878 (+ 220us) server_negotiation.cc:407] Connection header received
0805 22:42:38.126050 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.126051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.126065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.126099 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.126978 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.127703 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.128880 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.129195 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.129264 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.132794 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.132796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.132797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.132810 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.137548 (+ 4738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.137923 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.137924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.137925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.137981 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.138192 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.138193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.138193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.138339 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.138386 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.138582 (+ 196us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.138657 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:42:38.139309 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.124457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.23.197.66:45849)
0805 22:42:38.124929 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.124939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.125922 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.126175 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.126179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.126582 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:38.126781 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.126792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.129327 (+ 2535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.129328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.130016 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.130019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.130504 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.130506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.130573 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.131077 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:38.131083 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:38.133013 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:38.137460 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.137461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.137462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.137696 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.137909 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.137910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.137911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.137972 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.138282 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:38.138283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:38.138336 (+ 53us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.138811 (+ 475us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.139044 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260805 22:42:38.139307 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.124457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.23.197.66:44617)
0805 22:42:38.124709 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.124756 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.125918 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.126175 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.126180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.126796 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:38.126858 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.126860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.127793 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.127794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.128790 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.128793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.129286 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.129286 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.129434 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.130198 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:38.130209 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:38.132589 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:38.137632 (+ 5043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.137633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.137634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.137835 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.138057 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.138057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.138057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.138107 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.138436 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:38.138436 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:38.138470 (+ 34us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.138831 (+ 361us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.139090 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":148,"thread_start_us":107,"threads_started":1}
I20260805 22:42:38.139873 2534 heartbeater.cc:347] Connected to a master server at 127.23.197.124:40965
I20260805 22:42:38.139978 2534 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:38.140132 2534 heartbeater.cc:511] Master 127.23.197.124:40965 requested a full tablet report, sending...
I20260805 22:42:38.140153 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.124457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.66:36959)
0805 22:42:38.124952 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.124962 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.125923 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.126176 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.126180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.126589 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:38.126783 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.126790 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.127530 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.127531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.128791 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.128793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.129284 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.129286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.129434 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.130259 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:38.130264 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:38.133013 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:38.136751 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.136781 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.136783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.137162 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.137425 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.137426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.137426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.137576 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.137943 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:38.137951 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:38.138142 (+ 191us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.139695 (+ 1553us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.139838 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":329,"thread_start_us":79,"threads_started":1}
I20260805 22:42:38.140591 2076 ts_manager.cc:194] Registered new tserver with Master: 9c983822c3b841ff9f980aefcfb0a189 (127.23.197.66:42627)
I20260805 22:42:38.142087 2537 heartbeater.cc:347] Connected to a master server at 127.23.197.125:33535
I20260805 22:42:38.142146 2537 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:38.142230 2537 heartbeater.cc:511] Master 127.23.197.125:33535 requested a full tablet report, sending...
I20260805 22:42:38.142263 2536 heartbeater.cc:347] Connected to a master server at 127.23.197.126:37131
I20260805 22:42:38.142298 2536 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:38.142372 2536 heartbeater.cc:511] Master 127.23.197.126:37131 requested a full tablet report, sending...
I20260805 22:42:38.142627 1813 ts_manager.cc:194] Registered new tserver with Master: 9c983822c3b841ff9f980aefcfb0a189 (127.23.197.66:42627)
I20260805 22:42:38.143150 930 ts_manager.cc:194] Re-registered known tserver with Master: 9c983822c3b841ff9f980aefcfb0a189 (127.23.197.66:42627)
I20260805 22:42:38.143630 930 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:36959
I20260805 22:42:38.145484 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:45787
--local_ip_for_outbound_sockets=127.23.197.67
--tserver_master_addrs=127.23.197.126:37131
--webserver_port=34539
--webserver_interface=127.23.197.67
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab
--principal=oryx/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.197.126:37131,127.23.197.125:33535
--tserver_master_addrs=127.23.197.126:37131,127.23.197.125:33535,127.23.197.124:40965 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:42:38.213840 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.211124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40232 (local address 127.23.197.124:40965)
0805 22:42:38.211212 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.211214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.211224 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:38.211260 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.211261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.211283 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.211355 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:38.211504 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.211997 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.213014 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.213453 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.213545 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.213642 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.213723 (+ 81us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.213758 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 22:42:38.213840 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.211029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.0.0.1:40232)
0805 22:42:38.211092 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.211110 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.211173 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.211360 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.211361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.211363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:38.211418 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.211420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.212086 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.212087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.212923 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.212926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.213578 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.213580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.213646 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.213651 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.213716 (+ 65us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.213774 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260805 22:42:38.214385 2090 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260805 22:42:38.214547 2131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0aa186bced34b6d9599b9126c511cbd. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:38.214669 2131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: This master's current role is: LEARNER
W20260805 22:42:38.230800 2544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:38.231060 2544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:38.231086 2544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:38.232327 2544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:38.232369 2544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:38.232396 2544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:38.233914 2544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab
--principal=oryx/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:45787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=34539
--tserver_master_addrs=127.23.197.126:37131,127.23.197.125:33535,127.23.197.124:40965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:38.234108 2544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:38.234299 2544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:38.239750 2552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:38.239878 2544 server_base.cc:1034] running on GCE node
W20260805 22:42:38.239701 2551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:38.239714 2556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:38.240206 2544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:38.240497 2544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:38.241719 2544 hybrid_clock.cc:648] HybridClock initialized: now 1785969758241704 us; error 27 us; skew 500 ppm
Aug 05 22:42:38 dist-test-slave-g170 krb5kdc[869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969758, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:38.244712 2544 init.cc:377] Logged in from keytab as oryx/127.23.197.67@KRBTEST.COM (short username oryx)
I20260805 22:42:38.245280 2544 webserver.cc:460] Webserver started at http://127.23.197.67:34539/ using document root <none> and password file <none>
I20260805 22:42:38.245530 2544 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:38.245623 2544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:38.247200 2544 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:42:38.247977 2565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:38.248214 2544 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:42:38.248330 2544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "a644a7911c594902955424ff55f8ac67"
format_stamp: "Formatted at 2026-08-05 22:42:37 on dist-test-slave-g170"
server_key: "fa8216f4a433aade67a21047e77b470b"
server_key_iv: "34fedc74994a9f02adfde4f6be9394c9"
server_key_version: "encryptionkey@0"
I20260805 22:42:38.248701 2544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:38.260546 2544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:38.263209 2544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:38.263377 2544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:38.263636 2544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:38.264082 2544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:38.264135 2544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:38.264178 2544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:38.264205 2544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:38.271855 2544 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:45787
I20260805 22:42:38.271891 2720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:45787 every 8 connection(s)
I20260805 22:42:38.272310 2544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:38.281270 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 2544
Aug 05 22:42:38 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969758, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
Aug 05 22:42:38 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969758, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.124@KRBTEST.COM
Aug 05 22:42:38 dist-test-slave-g170 krb5kdc[869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969758, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.125@KRBTEST.COM
I20260805 22:42:38.290700 2727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.273361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.23.197.67:53709)
0805 22:42:38.273978 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.273986 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.275260 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.275570 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.275575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.276120 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:38.276419 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.276427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.277563 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.277564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.279103 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.279106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.279735 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.279736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.279815 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.280670 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:38.280675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:38.283112 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:38.288281 (+ 5169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.288282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.288283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.289078 (+ 795us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.289560 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.289560 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.289561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.289627 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.289915 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:38.289915 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:38.290041 (+ 126us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.290173 (+ 132us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.290414 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":139,"thread_start_us":65,"threads_started":1}
I20260805 22:42:38.291646 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.273271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.23.197.67:38697)
0805 22:42:38.273601 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.273718 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.275100 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.275700 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.275701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.276116 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:38.276417 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.276423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.277517 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.277518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.279103 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.279106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.279732 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.279733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.279815 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.280609 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:38.280614 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:38.283515 (+ 2901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:38.286277 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.286287 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.286290 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.286610 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.289166 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.289167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.289167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.289345 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.289840 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:38.289849 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:38.290041 (+ 192us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.291374 (+ 1333us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.291502 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":128,"spinlock_wait_cycles":8960,"thread_start_us":132,"threads_started":1}
I20260805 22:42:38.291766 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.283228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50824 (local address 127.23.197.125:33535)
0805 22:42:38.283352 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.283354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.283361 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:42:38.283383 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.283384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.283397 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.283667 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.284238 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.285048 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.286111 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.286423 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.286465 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.288516 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.288518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.288518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.288526 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.290355 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.290793 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.290794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.290794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.290852 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.291066 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.291067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.291067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.291213 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.291296 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.291548 (+ 252us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.291648 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260805 22:42:38.292383 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.273270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:53709 (local address 127.23.197.124:40965)
0805 22:42:38.273323 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.273325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.274047 (+ 722us) server_negotiation.cc:407] Connection header received
0805 22:42:38.275366 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.275368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.275380 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.275412 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.276550 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.277400 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.279237 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.279576 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.279625 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.283356 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.283358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.283359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.283369 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.288189 (+ 4820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.289444 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.289445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.289445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.289505 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.289699 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.289699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.289700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.289831 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.289891 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.292174 (+ 2283us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.292276 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:42:38.292837 2721 heartbeater.cc:347] Connected to a master server at 127.23.197.124:40965
I20260805 22:42:38.292868 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.273601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:38697 (local address 127.23.197.125:33535)
0805 22:42:38.273650 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.273652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.273851 (+ 199us) server_negotiation.cc:407] Connection header received
0805 22:42:38.275257 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.275259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.275274 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.275367 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.276533 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.277357 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.279237 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.279576 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.279630 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.283611 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.283612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.283613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.283620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.286176 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.288289 (+ 2113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.288291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.288291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.288367 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.289586 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.289587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.289587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.289736 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.289811 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.292654 (+ 2843us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.292733 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260805 22:42:38.292951 2721 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:38.293126 2735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.282543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.23.197.126:37131)
0805 22:42:38.283072 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.283074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.284122 (+ 1048us) server_negotiation.cc:407] Connection header received
0805 22:42:38.284183 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.284183 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.284196 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.284261 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.284954 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.285743 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.286951 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.287289 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.287354 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.288895 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.288896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.288897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.288904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.291299 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.291669 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.291670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.291670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.291728 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.291939 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.291939 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.291940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.292076 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.292852 (+ 776us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.292917 (+ 65us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.293031 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":113,"threads_started":1}
I20260805 22:42:38.293144 2723 heartbeater.cc:347] Connected to a master server at 127.23.197.125:33535
I20260805 22:42:38.293172 2721 heartbeater.cc:511] Master 127.23.197.124:40965 requested a full tablet report, sending...
I20260805 22:42:38.293183 2723 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:38.293262 2723 heartbeater.cc:511] Master 127.23.197.125:33535 requested a full tablet report, sending...
I20260805 22:42:38.293618 1813 ts_manager.cc:194] Registered new tserver with Master: a644a7911c594902955424ff55f8ac67 (127.23.197.67:45787)
I20260805 22:42:38.293658 2076 ts_manager.cc:194] Registered new tserver with Master: a644a7911c594902955424ff55f8ac67 (127.23.197.67:45787)
I20260805 22:42:38.295419 2728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.273273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.67:52957)
0805 22:42:38.273849 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.273859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.275221 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.275570 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.275575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.276116 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:38.276417 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.276423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.277517 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.277518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.279101 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.279104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.279631 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.279632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.279745 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.280322 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:38.280333 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:38.290420 (+ 10087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:38.294062 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.294063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.294064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.294282 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.294673 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:38.294674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:38.294674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:38.294742 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:38.295068 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:38.295069 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:38.295120 (+ 51us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.295207 (+ 87us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.295319 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":94,"thread_start_us":110,"threads_started":1}
I20260805 22:42:38.295450 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.273606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:52957 (local address 127.23.197.126:37131)
0805 22:42:38.273670 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.273673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.273908 (+ 235us) server_negotiation.cc:407] Connection header received
0805 22:42:38.275337 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.275338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.275352 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.275385 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.276533 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.277356 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.279224 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.279511 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.279550 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.290521 (+ 10971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.290523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.290523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.290531 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.292351 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.294539 (+ 2188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.294540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.294540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.294592 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.294839 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.294840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.294841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.294985 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.295068 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.295247 (+ 179us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.295342 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260805 22:42:38.296025 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.282670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.23.197.124:40965)
0805 22:42:38.283004 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.283006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.283280 (+ 274us) server_negotiation.cc:407] Connection header received
0805 22:42:38.283371 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.283373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.283386 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.283430 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.284842 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.285655 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.286864 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.287158 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.287227 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.289216 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.289217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.289217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.289225 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.294699 (+ 5474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.295107 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.295108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.295108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.295166 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.295409 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.295410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.295410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.295565 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.295643 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.295826 (+ 183us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.295916 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":61,"threads_started":1}
I20260805 22:42:38.295150 930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:38.297441 930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:38.297487 2722 heartbeater.cc:347] Connected to a master server at 127.23.197.126:37131
I20260805 22:42:38.297540 2722 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:38.297610 2722 heartbeater.cc:511] Master 127.23.197.126:37131 requested a full tablet report, sending...
I20260805 22:42:38.297945 929 ts_manager.cc:194] Re-registered known tserver with Master: a644a7911c594902955424ff55f8ac67 (127.23.197.67:45787)
I20260805 22:42:38.298338 2026 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: NON_VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: true } } }
I20260805 22:42:38.298452 2026 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:38.298475 929 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:52957
I20260805 22:42:38.299625 2026 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: attempting to promote NON_VOTER 3cd5e2e334ad4ed296b517a9335594f6 to VOTER
I20260805 22:42:38.299774 2026 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } }
I20260805 22:42:38.300254 2090 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 LEARNER]: Refusing update from remote peer c0aa186bced34b6d9599b9126c511cbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260805 22:42:38.300293 1828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Refusing update from remote peer c0aa186bced34b6d9599b9126c511cbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260805 22:42:38.300477 1763 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260805 22:42:38.300556 1764 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260805 22:42:38.301338 1763 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3cd5e2e334ad4ed296b517a9335594f6 (127.23.197.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:38.301491 1828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3cd5e2e334ad4ed296b517a9335594f6 (127.23.197.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:38.301573 2090 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3cd5e2e334ad4ed296b517a9335594f6 (127.23.197.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:38.302372 1763 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:38.302457 1764 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:38.302554 1764 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:38.302604 1764 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:38.302650 1764 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:38.302627 2025 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:38.302721 2025 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:42:38.302462 1763 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:38.302798 2131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0aa186bced34b6d9599b9126c511cbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:38.302881 2131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:42:38.308967 2755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.304862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.23.197.67:45787)
0805 22:42:38.305445 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.305450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.305460 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:38.305495 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.305497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.305511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.305708 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:38.305832 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.306953 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.308054 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.308561 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.308656 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.308740 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.308845 (+ 105us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.308890 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":159,"threads_started":1}
I20260805 22:42:38.308966 2753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.304739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:45787 (local address 127.0.0.1:59520)
0805 22:42:38.304922 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.304932 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.305185 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.305700 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.305701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.305704 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:38.305749 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.305751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.307062 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.307063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.307938 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.307941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.308658 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.308659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.308720 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.308725 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.308827 (+ 102us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.308876 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":76,"thread_start_us":147,"threads_started":1}
I20260805 22:42:38.309175 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.304655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:42627 (local address 127.0.0.1:34698)
0805 22:42:38.304717 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.304729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.304801 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.305304 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.305305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.305306 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:38.305358 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.305360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.307062 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.307063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.307959 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.307961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.308948 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.308949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.309007 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.309010 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.309039 (+ 29us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.309075 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":75}
I20260805 22:42:38.309641 2754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.304877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.23.197.66:42627)
0805 22:42:38.305118 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.305123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.305136 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:42:38.305199 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.305202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.305221 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.305782 (+ 561us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:38.305851 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.306953 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.308056 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.308849 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.309369 (+ 520us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.309496 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.309523 (+ 27us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.309566 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":119,"threads_started":1}
I20260805 22:42:38.310034 2632 tablet_service.cc:1467] Processing CreateTablet for tablet 1225e6e1256144eca25b50fd4d9b7446 (DEFAULT_TABLE table=test-table [id=f58ff7cb27bc40a79af9139252778dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:38.310426 2632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1225e6e1256144eca25b50fd4d9b7446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:38.310432 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.305202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33123 (local address 127.0.0.1:43398)
0805 22:42:38.305683 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.305693 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.305866 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.306180 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.306181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.306183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:38.306223 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.306225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.307041 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.307042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.309080 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.309082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.310195 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.310197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.310263 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.310268 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.310325 (+ 57us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.310364 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":795,"thread_start_us":96,"threads_started":1}
I20260805 22:42:38.310715 2757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.305446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.23.197.65:33123)
0805 22:42:38.305647 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.305653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.305859 (+ 206us) server_negotiation.cc:407] Connection header received
0805 22:42:38.305937 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.305939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.305954 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.306186 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:38.306284 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.306955 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.309170 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.310087 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.310179 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.310252 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.310340 (+ 88us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.310378 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":127,"threads_started":1}
I20260805 22:42:38.311285 2241 tablet_service.cc:1467] Processing CreateTablet for tablet 1225e6e1256144eca25b50fd4d9b7446 (DEFAULT_TABLE table=test-table [id=f58ff7cb27bc40a79af9139252778dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:38.311544 2241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1225e6e1256144eca25b50fd4d9b7446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:38.313849 2758 tablet_bootstrap.cc:492] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: Bootstrap starting.
I20260805 22:42:38.314412 2431 tablet_service.cc:1467] Processing CreateTablet for tablet 1225e6e1256144eca25b50fd4d9b7446 (DEFAULT_TABLE table=test-table [id=f58ff7cb27bc40a79af9139252778dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:38.314572 2762 tablet_bootstrap.cc:492] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: Bootstrap starting.
I20260805 22:42:38.314679 2758 tablet_bootstrap.cc:654] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:38.314819 2431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1225e6e1256144eca25b50fd4d9b7446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:38.315091 2758 log.cc:824] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:38.315461 2762 tablet_bootstrap.cc:654] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:38.315793 2762 log.cc:824] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:38.315825 2758 tablet_bootstrap.cc:492] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: No bootstrap required, opened a new log
I20260805 22:42:38.315902 2758 ts_tablet_manager.cc:1397] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:38.316665 2762 tablet_bootstrap.cc:492] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: No bootstrap required, opened a new log
I20260805 22:42:38.316736 2762 ts_tablet_manager.cc:1397] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:42:38.318367 2758 raft_consensus.cc:348] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } }
I20260805 22:42:38.318534 2758 raft_consensus.cc:374] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:38.318583 2758 raft_consensus.cc:729] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a644a7911c594902955424ff55f8ac67, State: Initialized, Role: FOLLOWER
I20260805 22:42:38.318727 2758 consensus_queue.cc:260] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } }
I20260805 22:42:38.319037 2758 ts_tablet_manager.cc:1428] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:42:38.319123 2722 heartbeater.cc:503] Master 127.23.197.126:37131 was elected leader, sending a full tablet report...
I20260805 22:42:38.319106 2767 tablet_bootstrap.cc:492] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: Bootstrap starting.
I20260805 22:42:38.319092 2762 raft_consensus.cc:348] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } }
I20260805 22:42:38.319258 2762 raft_consensus.cc:374] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:38.319293 2762 raft_consensus.cc:729] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0437f45cd9544ee0aacaeadd7a5b0f34, State: Initialized, Role: FOLLOWER
I20260805 22:42:38.319440 2762 consensus_queue.cc:260] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } }
I20260805 22:42:38.319701 2762 ts_tablet_manager.cc:1428] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:42:38.319803 2767 tablet_bootstrap.cc:654] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:38.319836 2331 heartbeater.cc:503] Master 127.23.197.126:37131 was elected leader, sending a full tablet report...
I20260805 22:42:38.320132 2767 log.cc:824] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:38.321022 2767 tablet_bootstrap.cc:492] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: No bootstrap required, opened a new log
I20260805 22:42:38.321089 2767 ts_tablet_manager.cc:1397] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:42:38.323364 2767 raft_consensus.cc:348] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } }
I20260805 22:42:38.323484 2767 raft_consensus.cc:374] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:38.323505 2767 raft_consensus.cc:729] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c983822c3b841ff9f980aefcfb0a189, State: Initialized, Role: FOLLOWER
I20260805 22:42:38.323612 2767 consensus_queue.cc:260] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } }
I20260805 22:42:38.323871 2767 ts_tablet_manager.cc:1428] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:42:38.323933 2536 heartbeater.cc:503] Master 127.23.197.126:37131 was elected leader, sending a full tablet report...
W20260805 22:42:38.375048 2539 tablet.cc:2369] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:42:38.475210 2333 tablet.cc:2369] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:42:38.524025 2725 tablet.cc:2369] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:42:38.662889 2773 raft_consensus.cc:482] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:38.663129 2773 raft_consensus.cc:504] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } }
I20260805 22:42:38.663627 2773 leader_election.cc:288] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a644a7911c594902955424ff55f8ac67 (127.23.197.67:45787), 0437f45cd9544ee0aacaeadd7a5b0f34 (127.23.197.65:33123)
I20260805 22:42:38.668185 2757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.664193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:55559 (local address 127.23.197.65:33123)
0805 22:42:38.664243 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.664245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.664255 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:38.664317 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.664318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.664332 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.664387 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:38.664570 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.665452 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.667162 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.667741 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.667831 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.667841 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.668052 (+ 211us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.668105 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260805 22:42:38.668198 2755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.663969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:36311 (local address 127.23.197.67:45787)
0805 22:42:38.664043 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.664045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.664118 (+ 73us) server_negotiation.cc:407] Connection header received
0805 22:42:38.664275 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.664276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.664288 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.664321 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:38.664572 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.665447 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.667162 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.667749 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.667831 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.667841 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.668071 (+ 230us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.668125 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 22:42:38.668473 2776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.664094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33123 (local address 127.23.197.66:55559)
0805 22:42:38.664199 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.664208 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.664254 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.664407 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.664408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.664410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:38.664483 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.664485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.665561 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.665563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.667023 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.667027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.667848 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.667849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.667932 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.667939 (+ 7us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.668309 (+ 370us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.668364 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":83,"thread_start_us":43,"threads_started":1}
I20260805 22:42:38.668494 2774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.663890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:45787 (local address 127.23.197.66:36311)
0805 22:42:38.664042 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:38.664055 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:38.664152 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:38.664411 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:38.664415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:38.664417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:38.664476 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.664478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.665561 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.665563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.667023 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:38.667027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.667847 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.667849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:38.667971 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.667975 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:38.668308 (+ 333us) client_negotiation.cc:240] Negotiation successful
0805 22:42:38.668364 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":73,"thread_start_us":68,"threads_started":1}
I20260805 22:42:38.669067 2269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1225e6e1256144eca25b50fd4d9b7446" candidate_uuid: "9c983822c3b841ff9f980aefcfb0a189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" is_pre_election: true
I20260805 22:42:38.669101 2636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1225e6e1256144eca25b50fd4d9b7446" candidate_uuid: "9c983822c3b841ff9f980aefcfb0a189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a644a7911c594902955424ff55f8ac67" is_pre_election: true
I20260805 22:42:38.669255 2269 raft_consensus.cc:2454] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c983822c3b841ff9f980aefcfb0a189 in term 0.
I20260805 22:42:38.669314 2636 raft_consensus.cc:2454] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c983822c3b841ff9f980aefcfb0a189 in term 0.
I20260805 22:42:38.669652 2370 leader_election.cc:302] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0437f45cd9544ee0aacaeadd7a5b0f34, 9c983822c3b841ff9f980aefcfb0a189; no voters:
I20260805 22:42:38.669834 2773 raft_consensus.cc:2781] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:38.669925 2773 raft_consensus.cc:482] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:38.669962 2773 raft_consensus.cc:3037] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:38.670730 2773 raft_consensus.cc:504] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } }
I20260805 22:42:38.670892 2773 leader_election.cc:288] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [CANDIDATE]: Term 1 election: Requested vote from peers a644a7911c594902955424ff55f8ac67 (127.23.197.67:45787), 0437f45cd9544ee0aacaeadd7a5b0f34 (127.23.197.65:33123)
I20260805 22:42:38.671068 2636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1225e6e1256144eca25b50fd4d9b7446" candidate_uuid: "9c983822c3b841ff9f980aefcfb0a189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a644a7911c594902955424ff55f8ac67"
I20260805 22:42:38.671101 2269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1225e6e1256144eca25b50fd4d9b7446" candidate_uuid: "9c983822c3b841ff9f980aefcfb0a189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34"
I20260805 22:42:38.671172 2636 raft_consensus.cc:3037] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:38.671166 2269 raft_consensus.cc:3037] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:38.671895 2636 raft_consensus.cc:2454] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c983822c3b841ff9f980aefcfb0a189 in term 1.
I20260805 22:42:38.671895 2269 raft_consensus.cc:2454] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c983822c3b841ff9f980aefcfb0a189 in term 1.
I20260805 22:42:38.672117 2370 leader_election.cc:302] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0437f45cd9544ee0aacaeadd7a5b0f34, 9c983822c3b841ff9f980aefcfb0a189; no voters:
I20260805 22:42:38.672247 2773 raft_consensus.cc:2781] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:38.672397 2773 raft_consensus.cc:686] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 LEADER]: Becoming Leader. State: Replica: 9c983822c3b841ff9f980aefcfb0a189, State: Running, Role: LEADER
I20260805 22:42:38.672590 2773 consensus_queue.cc:237] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } }
I20260805 22:42:38.673609 930 catalog_manager.cc:5495] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c983822c3b841ff9f980aefcfb0a189 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "9c983822c3b841ff9f980aefcfb0a189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:38.756636 2754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.754132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.23.197.66:42627)
0805 22:42:38.754200 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.754202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.754227 (+ 25us) server_negotiation.cc:407] Connection header received
0805 22:42:38.754272 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.754273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.754293 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.754356 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:38.754512 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.755089 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.755828 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.756032 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.756093 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.756279 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:38.756390 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:38.756460 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.756498 (+ 38us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.756518 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 22:42:38.758873 2269 raft_consensus.cc:1261] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 1 FOLLOWER]: Refusing update from remote peer 9c983822c3b841ff9f980aefcfb0a189: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:38.758873 2636 raft_consensus.cc:1261] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [term 1 FOLLOWER]: Refusing update from remote peer 9c983822c3b841ff9f980aefcfb0a189: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:38.759378 2777 consensus_queue.cc:1035] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:38.759596 2773 consensus_queue.cc:1035] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:38.762012 2780 mvcc.cc:205] Tried to move back new op lower bound from 7315332131872825344 to 7315332131523670016. Current Snapshot: MvccSnapshot[applied={T|T < 7315332131872825344}]
I20260805 22:42:38.763363 2782 mvcc.cc:205] Tried to move back new op lower bound from 7315332131872825344 to 7315332131523670016. Current Snapshot: MvccSnapshot[applied={T|T < 7315332131872825344}]
I20260805 22:42:38.764475 2781 mvcc.cc:205] Tried to move back new op lower bound from 7315332131872825344 to 7315332131523670016. Current Snapshot: MvccSnapshot[applied={T|T < 7315332131872825344 or (T in {7315332131872825344})}]
I20260805 22:42:38.764725 930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:38.764843 930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:38.766285 930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0aa186bced34b6d9599b9126c511cbd: Sending DeleteTablet for 3 replicas of tablet 1225e6e1256144eca25b50fd4d9b7446
I20260805 22:42:38.766688 2431 tablet_service.cc:1514] Processing DeleteTablet for tablet 1225e6e1256144eca25b50fd4d9b7446 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:38 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:34698
I20260805 22:42:38.767273 2789 tablet_replica.cc:330] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: stopping tablet replica
I20260805 22:42:38.767383 2789 raft_consensus.cc:2229] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:38.767482 2789 raft_consensus.cc:2258] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:38.767520 2632 tablet_service.cc:1514] Processing DeleteTablet for tablet 1225e6e1256144eca25b50fd4d9b7446 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:38 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:59520
I20260805 22:42:38.767792 2241 tablet_service.cc:1514] Processing DeleteTablet for tablet 1225e6e1256144eca25b50fd4d9b7446 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:38 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:43398
I20260805 22:42:38.767969 2792 tablet_replica.cc:330] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: stopping tablet replica
I20260805 22:42:38.768302 2792 raft_consensus.cc:2229] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:38.768487 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 884
I20260805 22:42:38.768514 2792 raft_consensus.cc:2258] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:38.768795 2794 tablet_replica.cc:330] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: stopping tablet replica
I20260805 22:42:38.768898 2794 raft_consensus.cc:2229] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:38.769001 2794 raft_consensus.cc:2258] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:38.772753 2794 ts_tablet_manager.cc:1905] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:38.774997 2794 ts_tablet_manager.cc:1918] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:38.775101 2794 log.cc:1196] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/1225e6e1256144eca25b50fd4d9b7446
I20260805 22:42:38.775492 2794 ts_tablet_manager.cc:1939] T 1225e6e1256144eca25b50fd4d9b7446 P 0437f45cd9544ee0aacaeadd7a5b0f34: Deleting consensus metadata
I20260805 22:42:38.776808 2792 ts_tablet_manager.cc:1905] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:38.778837 2792 ts_tablet_manager.cc:1918] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:38.779019 2792 log.cc:1196] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/1225e6e1256144eca25b50fd4d9b7446
I20260805 22:42:38.779403 2792 ts_tablet_manager.cc:1939] T 1225e6e1256144eca25b50fd4d9b7446 P a644a7911c594902955424ff55f8ac67: Deleting consensus metadata
I20260805 22:42:38.779901 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.767110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40240 (local address 127.23.197.124:40965)
0805 22:42:38.767169 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.767171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.767673 (+ 502us) server_negotiation.cc:407] Connection header received
0805 22:42:38.767743 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.767744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.767757 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.767809 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.768709 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.769402 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.772509 (+ 3107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.773867 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.773973 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.776492 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.776494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.776494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.776504 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.778402 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.778895 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.778897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.778897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.778964 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.779202 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.779203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.779204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.779362 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.779446 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.779701 (+ 255us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.779800 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 22:42:38.785358 2789 ts_tablet_manager.cc:1905] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:38.788476 2789 ts_tablet_manager.cc:1918] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:38.788575 2789 log.cc:1196] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/1225e6e1256144eca25b50fd4d9b7446
I20260805 22:42:38.788962 2789 ts_tablet_manager.cc:1939] T 1225e6e1256144eca25b50fd4d9b7446 P 9c983822c3b841ff9f980aefcfb0a189: Deleting consensus metadata
W20260805 22:42:38.789819 2170 connection.cc:535] server connection from 127.0.0.1:43398 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260805 22:42:38.789861 2569 connection.cc:535] server connection from 127.0.0.1:59520 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260805 22:42:38.790112 2370 connection.cc:535] server connection from 127.0.0.1:34698 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 22:42:38.799706 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.791829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40242 (local address 127.23.197.124:40965)
0805 22:42:38.791878 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.791879 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.792081 (+ 202us) server_negotiation.cc:407] Connection header received
0805 22:42:38.792253 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.792254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.792268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.792301 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.793576 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.794095 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.794884 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.795089 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.795165 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.796678 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.796679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.796680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.796687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.798416 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.798828 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.798829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.798830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.798898 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.799109 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.799110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.799111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.799270 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.799369 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.799514 (+ 145us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.799603 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260805 22:42:38.800069 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:38.791638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50830 (local address 127.23.197.125:33535)
0805 22:42:38.791695 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:38.791698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:38.792020 (+ 322us) server_negotiation.cc:407] Connection header received
0805 22:42:38.792161 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:38.792163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:38.792175 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:38.792235 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:38.793299 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.794164 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.795251 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:38.795553 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:38.795593 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:38.797100 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:38.797101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:38.797101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:38.797109 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:38.798796 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.799164 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.799165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.799166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.799219 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:38.799438 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:38.799440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:38.799441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:38.799574 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:38.799664 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:38.799883 (+ 219us) server_negotiation.cc:299] Negotiation successful
0805 22:42:38.799963 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:42:39.763957 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:39.763588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.67:41995)
0805 22:42:39.763752 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:39.763884 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
W20260805 22:42:39.764268 2722 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:37131 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
I20260805 22:42:39.765098 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:39.764687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.66:45103)
0805 22:42:39.764968 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:39.765049 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":96,"threads_started":1}
W20260805 22:42:39.765280 2536 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:37131 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
I20260805 22:42:39.765316 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:39.764914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.65:44261)
0805 22:42:39.765184 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:39.765268 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":66,"threads_started":1}
W20260805 22:42:39.765503 2331 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:37131 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
I20260805 22:42:40.267169 2807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c0aa186bced34b6d9599b9126c511cbd)
I20260805 22:42:40.267335 2807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } }
I20260805 22:42:40.267642 2807 leader_election.cc:288] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c0aa186bced34b6d9599b9126c511cbd (127.23.197.126:37131), c659c7f9c3be4fbab4c13e9de9e26bb8 (127.23.197.125:33535)
I20260805 22:42:40.268213 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.267858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34700)
0805 22:42:40.268079 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.268145 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":67,"threads_started":1}
W20260805 22:42:40.268504 2052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
I20260805 22:42:40.268873 2811 raft_consensus.cc:482] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c0aa186bced34b6d9599b9126c511cbd)
I20260805 22:42:40.269052 2811 raft_consensus.cc:504] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } }
I20260805 22:42:40.269325 2811 leader_election.cc:288] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c0aa186bced34b6d9599b9126c511cbd (127.23.197.126:37131), 3cd5e2e334ad4ed296b517a9335594f6 (127.23.197.124:40965)
I20260805 22:42:40.269680 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.269548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34718)
0805 22:42:40.269611 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.269641 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 22:42:40.269778 2052 leader_election.cc:334] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c0aa186bced34b6d9599b9126c511cbd (127.23.197.126:37131): Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
I20260805 22:42:40.271543 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.267899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:33535 (local address 127.0.0.1:50846)
0805 22:42:40.268196 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.268209 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:40.268288 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:40.268586 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:40.268588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:40.268590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:40.268672 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.268675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.269691 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.269692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.270710 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.270713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.271285 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.271286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.271363 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.271367 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:40.271428 (+ 61us) client_negotiation.cc:240] Negotiation successful
0805 22:42:40.271473 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":62,"threads_started":1}
I20260805 22:42:40.271544 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.268079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50846 (local address 127.23.197.125:33535)
0805 22:42:40.268391 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:40.268394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:40.268407 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:42:40.268444 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:40.268445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:40.268465 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:40.268560 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:40.268794 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.269593 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.270800 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.271192 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.271277 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.271287 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:40.271434 (+ 147us) server_negotiation.cc:299] Negotiation successful
0805 22:42:40.271483 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":45,"threads_started":1}
I20260805 22:42:40.272471 1828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cd5e2e334ad4ed296b517a9335594f6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" is_pre_election: true
I20260805 22:42:40.272614 1828 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cd5e2e334ad4ed296b517a9335594f6 in term 1.
I20260805 22:42:40.272820 2055 leader_election.cc:302] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3cd5e2e334ad4ed296b517a9335594f6, c659c7f9c3be4fbab4c13e9de9e26bb8; no voters: c0aa186bced34b6d9599b9126c511cbd
I20260805 22:42:40.272966 2807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260805 22:42:40.273005 2807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c0aa186bced34b6d9599b9126c511cbd)
I20260805 22:42:40.273037 2807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 1 FOLLOWER]: Advancing to term 2
I20260805 22:42:40.273388 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.270055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34704)
0805 22:42:40.273248 (+ 3193us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.273344 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3175,"thread_start_us":89,"threads_started":1}
W20260805 22:42:40.273530 1793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
I20260805 22:42:40.274245 2807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } }
I20260805 22:42:40.274423 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.274316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34720)
0805 22:42:40.274364 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.274394 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260805 22:42:40.274530 1793 leader_election.cc:334] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c0aa186bced34b6d9599b9126c511cbd (127.23.197.126:37131): Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
I20260805 22:42:40.274636 1828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cd5e2e334ad4ed296b517a9335594f6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8"
I20260805 22:42:40.274724 1828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 1 FOLLOWER]: Advancing to term 2
I20260805 22:42:40.275100 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.274464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34726)
0805 22:42:40.274970 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.275059 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462}
I20260805 22:42:40.275584 1828 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cd5e2e334ad4ed296b517a9335594f6 in term 2.
I20260805 22:42:40.275794 2055 leader_election.cc:302] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd5e2e334ad4ed296b517a9335594f6, c659c7f9c3be4fbab4c13e9de9e26bb8; no voters:
I20260805 22:42:40.275969 2807 leader_election.cc:288] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [CANDIDATE]: Term 2 election: Requested vote from peers c0aa186bced34b6d9599b9126c511cbd (127.23.197.126:37131), c659c7f9c3be4fbab4c13e9de9e26bb8 (127.23.197.125:33535)
I20260805 22:42:40.276027 2807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 2 FOLLOWER]: Leader election won for term 2
I20260805 22:42:40.276059 2807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [term 2 LEADER]: Becoming Leader. State: Replica: 3cd5e2e334ad4ed296b517a9335594f6, State: Running, Role: LEADER
I20260805 22:42:40.276116 2807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } }
I20260805 22:42:40.276424 2807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cd5e2e334ad4ed296b517a9335594f6. Latest consensus state: current_term: 2 leader_uuid: "3cd5e2e334ad4ed296b517a9335594f6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:40.276494 2807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:40.276662 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.275914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34730)
0805 22:42:40.276611 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.276640 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":673}
W20260805 22:42:40.276780 2052 leader_election.cc:334] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c0aa186bced34b6d9599b9126c511cbd (127.23.197.126:37131): Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
I20260805 22:42:40.276866 2821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:40.277379 2821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:40.277505 2821 catalog_manager.cc:1259] Loaded cluster ID: 989888fdf71e43e4b1459dcdf7f54a1b
I20260805 22:42:40.277545 2821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:40.277601 2814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.269938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:40965 (local address 127.0.0.1:40248)
0805 22:42:40.270374 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.270387 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:40.270475 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:40.274010 (+ 3535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:40.274012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:40.274016 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:40.274093 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.274096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.275259 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.275261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.276650 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.276653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.277339 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.277341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.277404 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.277413 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 22:42:40.277478 (+ 65us) client_negotiation.cc:240] Negotiation successful
0805 22:42:40.277523 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":328,"threads_started":1}
I20260805 22:42:40.277612 2816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.270246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.23.197.124:40965)
0805 22:42:40.273833 (+ 3587us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:40.273835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:40.273847 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:42:40.273874 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:40.273875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:40.273888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:40.273984 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:40.274252 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.275159 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.276790 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.277243 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.277329 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.277340 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:40.277483 (+ 143us) server_negotiation.cc:299] Negotiation successful
0805 22:42:40.277525 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3552,"thread_start_us":95,"threads_started":1}
I20260805 22:42:40.277792 2821 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:40.277904 2821 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Loaded TSK: 0
I20260805 22:42:40.278091 2821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:42:40.278182 2090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3cd5e2e334ad4ed296b517a9335594f6" is_pre_election: true
I20260805 22:42:40.278494 1794 leader_election.cc:302] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c659c7f9c3be4fbab4c13e9de9e26bb8; no voters: 3cd5e2e334ad4ed296b517a9335594f6, c0aa186bced34b6d9599b9126c511cbd
I20260805 22:42:40.278667 2811 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:42:40.287561 2074 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:40.289592 2074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:40.290860 1828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [term 2 FOLLOWER]: Refusing update from remote peer 3cd5e2e334ad4ed296b517a9335594f6: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260805 22:42:40.290959 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.290835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34748)
0805 22:42:40.290886 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.290928 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260805 22:42:40.291131 2819 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260805 22:42:40.291383 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.291229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.0.0.1:34762)
0805 22:42:40.291322 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.291352 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260805 22:42:40.291478 2052 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 -> Peer c0aa186bced34b6d9599b9126c511cbd (127.23.197.126:37131): Couldn't send request to peer c0aa186bced34b6d9599b9126c511cbd. Status: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260805 22:42:40.291810 2811 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cd5e2e334ad4ed296b517a9335594f6. Latest consensus state: current_term: 2 leader_uuid: "3cd5e2e334ad4ed296b517a9335594f6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:40.291929 2811 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:42:40.292376 2822 mvcc.cc:205] Tried to move back new op lower bound from 7315332138148487168 to 7315332138091384832. Current Snapshot: MvccSnapshot[applied={T|T < 7315332138148487168}]
I20260805 22:42:40.292474 2807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3cd5e2e334ad4ed296b517a9335594f6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:40.292560 2807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:40.292788 2823 mvcc.cc:205] Tried to move back new op lower bound from 7315332138148487168 to 7315332138091384832. Current Snapshot: MvccSnapshot[applied={T|T < 7315332138148487168}]
I20260805 22:42:40.293274 2811 sys_catalog.cc:455] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3cd5e2e334ad4ed296b517a9335594f6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c0aa186bced34b6d9599b9126c511cbd" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37131 } } peers { permanent_uuid: "c659c7f9c3be4fbab4c13e9de9e26bb8" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 33535 } attrs { promote: false } } peers { permanent_uuid: "3cd5e2e334ad4ed296b517a9335594f6" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 40965 } attrs { promote: false } } }
I20260805 22:42:40.293409 2811 sys_catalog.cc:458] T 00000000000000000000000000000000 P c659c7f9c3be4fbab4c13e9de9e26bb8 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:42:40.298576 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.294898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:42627 (local address 127.0.0.1:34726)
0805 22:42:40.294988 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.294997 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:40.295048 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:40.295544 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:40.295545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:40.295550 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:40.295633 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.295635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.296621 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.296622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.297650 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.297653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.298351 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.298352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.298423 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.298427 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:40.298457 (+ 30us) client_negotiation.cc:240] Negotiation successful
0805 22:42:40.298497 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":34}
I20260805 22:42:40.298677 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.294859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33123 (local address 127.0.0.1:43400)
0805 22:42:40.294909 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.294921 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:40.294966 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:40.295504 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:40.295505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:40.295507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:40.295566 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.295568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.296308 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.296310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.297604 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.297608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.298403 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.298405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.298471 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.298475 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:40.298562 (+ 87us) client_negotiation.cc:240] Negotiation successful
0805 22:42:40.298607 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260805 22:42:40.298717 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.294943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.23.197.65:33123)
0805 22:42:40.295330 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:40.295333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:40.295343 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:40.295398 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:40.295399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:40.295410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:40.295502 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:40.295640 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.296219 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.297727 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.298280 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.298403 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.298420 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:40.298573 (+ 153us) server_negotiation.cc:299] Negotiation successful
0805 22:42:40.298635 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":76,"threads_started":1}
I20260805 22:42:40.298970 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.295129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.23.197.66:42627)
0805 22:42:40.295407 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:40.295409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:40.295418 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:40.295442 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:40.295443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:40.295453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:40.295575 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:40.295746 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.296510 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.297741 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.298241 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.298800 (+ 559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.298818 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:40.298840 (+ 22us) server_negotiation.cc:299] Negotiation successful
0805 22:42:40.298893 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":80,"threads_started":1}
I20260805 22:42:40.299026 2241 tablet_service.cc:1467] Processing CreateTablet for tablet a05656aa31284bed944c2b564e3dfe80 (DEFAULT_TABLE table=test-table [id=ae1cd93b80da4964a584905a8b9e9f7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:40.299201 2241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05656aa31284bed944c2b564e3dfe80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:40.299183 2431 tablet_service.cc:1467] Processing CreateTablet for tablet a05656aa31284bed944c2b564e3dfe80 (DEFAULT_TABLE table=test-table [id=ae1cd93b80da4964a584905a8b9e9f7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:40.299311 2431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05656aa31284bed944c2b564e3dfe80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:40.299347 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.294986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:45787 (local address 127.0.0.1:59528)
0805 22:42:40.295130 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.295145 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:40.295188 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:40.295544 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:40.295545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:40.295547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:40.295633 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.295635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.296595 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.296596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.297903 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:40.297905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.299093 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.299095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:40.299163 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.299167 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:40.299202 (+ 35us) client_negotiation.cc:240] Negotiation successful
0805 22:42:40.299258 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":30,"thread_start_us":88,"threads_started":1}
I20260805 22:42:40.299620 2835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.295269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.23.197.67:45787)
0805 22:42:40.295392 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:40.295395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:40.295404 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:40.295433 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:40.295434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:40.295445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:40.295702 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:40.295745 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.296515 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.297994 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.298576 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.299453 (+ 877us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.299471 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:40.299492 (+ 21us) server_negotiation.cc:299] Negotiation successful
0805 22:42:40.299538 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260805 22:42:40.300935 2837 tablet_bootstrap.cc:492] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: Bootstrap starting.
I20260805 22:42:40.301039 2632 tablet_service.cc:1467] Processing CreateTablet for tablet a05656aa31284bed944c2b564e3dfe80 (DEFAULT_TABLE table=test-table [id=ae1cd93b80da4964a584905a8b9e9f7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:40.301100 2838 tablet_bootstrap.cc:492] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34: Bootstrap starting.
I20260805 22:42:40.301214 2632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05656aa31284bed944c2b564e3dfe80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:40.301749 2838 tablet_bootstrap.cc:654] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:40.302724 2838 tablet_bootstrap.cc:492] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34: No bootstrap required, opened a new log
I20260805 22:42:40.302876 2838 ts_tablet_manager.cc:1397] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:42:40.302922 2837 tablet_bootstrap.cc:654] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:40.303105 2843 tablet_bootstrap.cc:492] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: Bootstrap starting.
I20260805 22:42:40.303395 2838 raft_consensus.cc:348] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } }
I20260805 22:42:40.303493 2838 raft_consensus.cc:374] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:40.303509 2838 raft_consensus.cc:729] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0437f45cd9544ee0aacaeadd7a5b0f34, State: Initialized, Role: FOLLOWER
I20260805 22:42:40.303506 2843 tablet_bootstrap.cc:654] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:40.303578 2838 consensus_queue.cc:260] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } }
I20260805 22:42:40.303766 2838 ts_tablet_manager.cc:1428] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 22:42:40.304162 2843 tablet_bootstrap.cc:492] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: No bootstrap required, opened a new log
I20260805 22:42:40.304273 2843 ts_tablet_manager.cc:1397] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:40.304291 2837 tablet_bootstrap.cc:492] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: No bootstrap required, opened a new log
I20260805 22:42:40.304340 2837 ts_tablet_manager.cc:1397] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260805 22:42:40.304447 2330 heartbeater.cc:503] Master 127.23.197.124:40965 was elected leader, sending a full tablet report...
I20260805 22:42:40.304423 2843 raft_consensus.cc:348] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } }
I20260805 22:42:40.304488 2843 raft_consensus.cc:374] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:40.304510 2843 raft_consensus.cc:729] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a644a7911c594902955424ff55f8ac67, State: Initialized, Role: FOLLOWER
I20260805 22:42:40.304484 2837 raft_consensus.cc:348] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } }
I20260805 22:42:40.304567 2837 raft_consensus.cc:374] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:40.304557 2843 consensus_queue.cc:260] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } }
I20260805 22:42:40.304593 2837 raft_consensus.cc:729] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c983822c3b841ff9f980aefcfb0a189, State: Initialized, Role: FOLLOWER
I20260805 22:42:40.304643 2837 consensus_queue.cc:260] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } }
I20260805 22:42:40.304750 2843 ts_tablet_manager.cc:1428] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:40.304822 2837 ts_tablet_manager.cc:1428] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:40.305737 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.305485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.66:37629)
0805 22:42:40.305648 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.305693 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260805 22:42:40.305919 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.305152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.67:48983)
0805 22:42:40.305765 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.305885 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":92,"threads_started":1}
I20260805 22:42:40.306103 2534 heartbeater.cc:503] Master 127.23.197.124:40965 was elected leader, sending a full tablet report...
I20260805 22:42:40.306468 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.304017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.65:42517)
0805 22:42:40.306390 (+ 2373us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.306428 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2347,"thread_start_us":98,"threads_started":1}
I20260805 22:42:40.306470 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.306248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.66:36619)
0805 22:42:40.306404 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.306439 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260805 22:42:40.306687 2721 heartbeater.cc:503] Master 127.23.197.124:40965 was elected leader, sending a full tablet report...
I20260805 22:42:40.306926 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.306834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.67:42717)
0805 22:42:40.306867 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.306893 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260805 22:42:40.306846 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.306727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.65:35553)
0805 22:42:40.306786 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.306811 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260805 22:42:40.336825 2850 raft_consensus.cc:482] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:40.336922 2850 raft_consensus.cc:504] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } }
I20260805 22:42:40.337064 2850 leader_election.cc:288] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0437f45cd9544ee0aacaeadd7a5b0f34 (127.23.197.65:33123), a644a7911c594902955424ff55f8ac67 (127.23.197.67:45787)
I20260805 22:42:40.337396 2636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a05656aa31284bed944c2b564e3dfe80" candidate_uuid: "9c983822c3b841ff9f980aefcfb0a189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a644a7911c594902955424ff55f8ac67" is_pre_election: true
I20260805 22:42:40.337399 2269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a05656aa31284bed944c2b564e3dfe80" candidate_uuid: "9c983822c3b841ff9f980aefcfb0a189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" is_pre_election: true
I20260805 22:42:40.337509 2269 raft_consensus.cc:2454] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c983822c3b841ff9f980aefcfb0a189 in term 0.
I20260805 22:42:40.337509 2636 raft_consensus.cc:2454] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c983822c3b841ff9f980aefcfb0a189 in term 0.
I20260805 22:42:40.337672 2370 leader_election.cc:302] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0437f45cd9544ee0aacaeadd7a5b0f34, 9c983822c3b841ff9f980aefcfb0a189; no voters:
I20260805 22:42:40.337802 2850 raft_consensus.cc:2781] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:40.337855 2850 raft_consensus.cc:482] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:40.337877 2850 raft_consensus.cc:3037] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:40.338636 2850 raft_consensus.cc:504] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } }
I20260805 22:42:40.338776 2850 leader_election.cc:288] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [CANDIDATE]: Term 1 election: Requested vote from peers 0437f45cd9544ee0aacaeadd7a5b0f34 (127.23.197.65:33123), a644a7911c594902955424ff55f8ac67 (127.23.197.67:45787)
I20260805 22:42:40.338953 2269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a05656aa31284bed944c2b564e3dfe80" candidate_uuid: "9c983822c3b841ff9f980aefcfb0a189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34"
I20260805 22:42:40.339035 2269 raft_consensus.cc:3037] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:40.338966 2636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a05656aa31284bed944c2b564e3dfe80" candidate_uuid: "9c983822c3b841ff9f980aefcfb0a189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a644a7911c594902955424ff55f8ac67"
I20260805 22:42:40.339067 2636 raft_consensus.cc:3037] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:40.339630 2269 raft_consensus.cc:2454] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c983822c3b841ff9f980aefcfb0a189 in term 1.
I20260805 22:42:40.339797 2370 leader_election.cc:302] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0437f45cd9544ee0aacaeadd7a5b0f34, 9c983822c3b841ff9f980aefcfb0a189; no voters:
I20260805 22:42:40.339805 2636 raft_consensus.cc:2454] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c983822c3b841ff9f980aefcfb0a189 in term 1.
I20260805 22:42:40.339895 2850 raft_consensus.cc:2781] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:40.340087 2850 raft_consensus.cc:686] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 LEADER]: Becoming Leader. State: Replica: 9c983822c3b841ff9f980aefcfb0a189, State: Running, Role: LEADER
I20260805 22:42:40.340160 2850 consensus_queue.cc:237] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } }
I20260805 22:42:40.340612 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.340385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.66:38863)
0805 22:42:40.340516 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.340557 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260805 22:42:40.340947 2074 catalog_manager.cc:5495] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c983822c3b841ff9f980aefcfb0a189 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "9c983822c3b841ff9f980aefcfb0a189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c983822c3b841ff9f980aefcfb0a189" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:40.357128 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.354796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34728 (local address 127.23.197.66:42627)
0805 22:42:40.354847 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:40.354849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:40.354859 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:40.354934 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:40.354935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:40.354946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:40.355013 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:40.355148 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.355627 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.356414 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:40.356612 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:40.356684 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:40.356865 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:40.356899 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:40.356955 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:40.357011 (+ 56us) server_negotiation.cc:299] Negotiation successful
0805 22:42:40.357034 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260805 22:42:40.358227 2636 raft_consensus.cc:1261] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [term 1 FOLLOWER]: Refusing update from remote peer 9c983822c3b841ff9f980aefcfb0a189: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:40.358238 2269 raft_consensus.cc:1261] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 1 FOLLOWER]: Refusing update from remote peer 9c983822c3b841ff9f980aefcfb0a189: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:40.358508 2854 consensus_queue.cc:1035] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a644a7911c594902955424ff55f8ac67" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:40.358628 2850 consensus_queue.cc:1035] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0437f45cd9544ee0aacaeadd7a5b0f34" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:40.359972 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.359754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.65:36241)
0805 22:42:40.359887 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.359932 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260805 22:42:40.360287 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.359485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.67:34597)
0805 22:42:40.360212 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.360250 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":705}
I20260805 22:42:40.360710 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.360593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.66:51487)
0805 22:42:40.360637 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.360664 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260805 22:42:40.361734 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.361544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.66:35465)
0805 22:42:40.361663 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.361706 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260805 22:42:40.362152 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.362031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.67:41987)
0805 22:42:40.362094 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.362123 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260805 22:42:40.362391 2076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:40.362506 2076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:40.362859 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.362731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:37131 (local address 127.23.197.65:36167)
0805 22:42:40.362791 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.362828 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260805 22:42:40.364063 2076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cd5e2e334ad4ed296b517a9335594f6: Sending DeleteTablet for 3 replicas of tablet a05656aa31284bed944c2b564e3dfe80
I20260805 22:42:40.364456 2241 tablet_service.cc:1514] Processing DeleteTablet for tablet a05656aa31284bed944c2b564e3dfe80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:40 UTC) from {username='oryx', principal='oryx/127.23.197.124@KRBTEST.COM'} at 127.0.0.1:43400
I20260805 22:42:40.364439 2632 tablet_service.cc:1514] Processing DeleteTablet for tablet a05656aa31284bed944c2b564e3dfe80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:40 UTC) from {username='oryx', principal='oryx/127.23.197.124@KRBTEST.COM'} at 127.0.0.1:59528
I20260805 22:42:40.364497 2431 tablet_service.cc:1514] Processing DeleteTablet for tablet a05656aa31284bed944c2b564e3dfe80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:40 UTC) from {username='oryx', principal='oryx/127.23.197.124@KRBTEST.COM'} at 127.0.0.1:34726
I20260805 22:42:40.364682 2867 tablet_replica.cc:330] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34: stopping tablet replica
I20260805 22:42:40.364756 2867 raft_consensus.cc:2229] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:40.364823 2868 tablet_replica.cc:330] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: stopping tablet replica
I20260805 22:42:40.364842 2869 tablet_replica.cc:330] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: stopping tablet replica
I20260805 22:42:40.364895 2869 raft_consensus.cc:2229] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:40.364899 2868 raft_consensus.cc:2229] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:40.364948 2869 raft_consensus.cc:2258] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:40.364959 2868 raft_consensus.cc:2258] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:40.364974 2867 raft_consensus.cc:2258] T a05656aa31284bed944c2b564e3dfe80 P 0437f45cd9544ee0aacaeadd7a5b0f34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:40.365083 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 2022
I20260805 22:42:40.366140 2868 ts_tablet_manager.cc:1905] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:40.368631 2868 ts_tablet_manager.cc:1918] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:40.368723 2868 log.cc:1196] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/a05656aa31284bed944c2b564e3dfe80
I20260805 22:42:40.369102 2868 ts_tablet_manager.cc:1939] T a05656aa31284bed944c2b564e3dfe80 P 9c983822c3b841ff9f980aefcfb0a189: Deleting consensus metadata
I20260805 22:42:40.369541 2055 catalog_manager.cc:4841] TS 9c983822c3b841ff9f980aefcfb0a189 (127.23.197.66:42627): tablet a05656aa31284bed944c2b564e3dfe80 (table test-table [id=ae1cd93b80da4964a584905a8b9e9f7a]) successfully deleted
I20260805 22:42:40.382706 2869 ts_tablet_manager.cc:1905] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:40.385102 2869 ts_tablet_manager.cc:1918] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:40.385210 2869 log.cc:1196] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/a05656aa31284bed944c2b564e3dfe80
I20260805 22:42:40.385432 2869 ts_tablet_manager.cc:1939] T a05656aa31284bed944c2b564e3dfe80 P a644a7911c594902955424ff55f8ac67: Deleting consensus metadata
I20260805 22:42:40.385911 2053 catalog_manager.cc:4841] TS a644a7911c594902955424ff55f8ac67 (127.23.197.67:45787): tablet a05656aa31284bed944c2b564e3dfe80 (table test-table [id=ae1cd93b80da4964a584905a8b9e9f7a]) successfully deleted
I20260805 22:42:40.390348 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:40.390181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33123 (local address 127.0.0.1:43410)
0805 22:42:40.390262 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:40.390301 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260805 22:42:40.390326 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 2339
W20260805 22:42:40.394134 2053 catalog_manager.cc:4584] TS 0437f45cd9544ee0aacaeadd7a5b0f34 (127.23.197.65:33123): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a05656aa31284bed944c2b564e3dfe80: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33123: connect: Connection refused (error 111)
I20260805 22:42:40.403684 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 2544
I20260805 22:42:40.420680 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 1624
I20260805 22:42:40.434262 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 1888
2026-08-05T22:42:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5290 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:40 dist-test-slave-g170 krb5kdc[2875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:40 dist-test-slave-g170 krb5kdc[2875](info): set up 2 sockets
Aug 05 22:42:40 dist-test-slave-g170 krb5kdc[2875](info): commencing operation
krb5kdc: starting...
W20260805 22:42:41.701256 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:41 dist-test-slave-g170 krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.126.keytab.
Entry for principal oryx/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.126.keytab.
I20260805 22:42:41.828408 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:35613
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.126.keytab
--principal=oryx/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:35613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:41.913756 2893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:41.913996 2893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:41.914033 2893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:41.915155 2893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:41.915220 2893 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260805 22:42:41.915242 2893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:41.915264 2893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:41.915295 2893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:41.915318 2893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:41.916868 2893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:35613
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.126.keytab
--principal=oryx/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:35613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:41.917141 2893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:41.917367 2893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:41.922501 2899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:41.922477 2893 server_base.cc:1034] running on GCE node
W20260805 22:42:41.922435 2900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:41.922442 2904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:41.922952 2893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:41.923264 2893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:41.924467 2893 hybrid_clock.cc:648] HybridClock initialized: now 1785969761924451 us; error 31 us; skew 500 ppm
Aug 05 22:42:41 dist-test-slave-g170 krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969761, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:41.927268 2893 init.cc:377] Logged in from keytab as oryx/127.23.197.126@KRBTEST.COM (short username oryx)
I20260805 22:42:41.927734 2893 webserver.cc:460] Webserver started at http://127.23.197.126:35733/ using document root <none> and password file <none>
I20260805 22:42:41.927986 2893 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:41.928053 2893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:41.928200 2893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:41.929322 2893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "26d90433919d4510afad78ad906cf728"
format_stamp: "Formatted at 2026-08-05 22:42:41 on dist-test-slave-g170"
server_key: "ffed0f5ab81edd7fb3eebe3a25fc2055"
server_key_iv: "95ef8122bffc9b7dae93dd335e4d72b5"
server_key_version: "encryptionkey@0"
I20260805 22:42:41.929692 2893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "26d90433919d4510afad78ad906cf728"
format_stamp: "Formatted at 2026-08-05 22:42:41 on dist-test-slave-g170"
server_key: "ffed0f5ab81edd7fb3eebe3a25fc2055"
server_key_iv: "95ef8122bffc9b7dae93dd335e4d72b5"
server_key_version: "encryptionkey@0"
I20260805 22:42:41.931063 2893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:42:41.931823 2915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:41.932013 2893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:41.932103 2893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "26d90433919d4510afad78ad906cf728"
format_stamp: "Formatted at 2026-08-05 22:42:41 on dist-test-slave-g170"
server_key: "ffed0f5ab81edd7fb3eebe3a25fc2055"
server_key_iv: "95ef8122bffc9b7dae93dd335e4d72b5"
server_key_version: "encryptionkey@0"
I20260805 22:42:41.932185 2893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:41.963251 2893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:41.965680 2893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:41.965935 2893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:41.971050 2893 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:35613
I20260805 22:42:41.971120 3003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:35613 every 8 connection(s)
I20260805 22:42:41.971490 2893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:41.972230 3004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:41.974198 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 2893
I20260805 22:42:41.974305 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:41.974510 24341 external_mini_cluster.cc:1428] Setting key d5c725709234f75599c494100fd60a7f
I20260805 22:42:41.975060 3004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728: Bootstrap starting.
I20260805 22:42:41.975829 3004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:41.976162 3004 log.cc:824] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:41.977020 3004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728: No bootstrap required, opened a new log
I20260805 22:42:41.979290 3004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26d90433919d4510afad78ad906cf728" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35613 } }
I20260805 22:42:41.979427 3004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:41.979450 3004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26d90433919d4510afad78ad906cf728, State: Initialized, Role: FOLLOWER
I20260805 22:42:41.979611 3004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26d90433919d4510afad78ad906cf728" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35613 } }
I20260805 22:42:41.979697 3004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:41.979741 3004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:41.980078 3004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:41.980686 3004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26d90433919d4510afad78ad906cf728" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35613 } }
I20260805 22:42:41.980824 3004 leader_election.cc:302] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26d90433919d4510afad78ad906cf728; no voters:
I20260805 22:42:41.981015 3004 leader_election.cc:288] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:41.981041 3010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:41.981161 3010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [term 1 LEADER]: Becoming Leader. State: Replica: 26d90433919d4510afad78ad906cf728, State: Running, Role: LEADER
I20260805 22:42:41.981257 3010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26d90433919d4510afad78ad906cf728" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35613 } }
I20260805 22:42:41.981312 3004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:41.981587 3010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26d90433919d4510afad78ad906cf728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26d90433919d4510afad78ad906cf728" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35613 } } }
I20260805 22:42:41.981657 3010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:41.981796 3010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26d90433919d4510afad78ad906cf728. Latest consensus state: current_term: 1 leader_uuid: "26d90433919d4510afad78ad906cf728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26d90433919d4510afad78ad906cf728" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35613 } } }
I20260805 22:42:41.981869 3010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:41.982007 3017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 05 22:42:41 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:41.982569 3017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:41.985914 3017 catalog_manager.cc:1347] Generated new cluster ID: 3c4355f05dad43d98384930291fb4ca3
I20260805 22:42:41.985976 3017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:41.987154 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:41.975089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48136 (local address 127.23.197.126:35613)
0805 22:42:41.975285 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:41.975293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:41.975325 (+ 32us) server_negotiation.cc:407] Connection header received
0805 22:42:41.975916 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:41.975926 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:41.975987 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:41.976116 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:41.976939 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:41.977875 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:41.978991 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:41.979418 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:41.979551 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:41.982800 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:41.982806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:41.982807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:41.982822 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:41.985214 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:41.985748 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:41.985752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:41.985752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:41.985871 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:41.986089 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:41.986090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:41.986090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:41.986462 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:41.986633 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:41.986750 (+ 117us) server_negotiation.cc:299] Negotiation successful
0805 22:42:41.986913 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260805 22:42:41.996361 3017 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:41.996984 3017 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:42.002657 3017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728: Generated new TSK 0
I20260805 22:42:42.002825 3017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab.
Entry for principal oryx/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab.
I20260805 22:42:42.036037 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:35613
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab
--principal=oryx/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:42.120896 3036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:42.121124 3036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:42.121169 3036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:42.122310 3036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:42.122347 3036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:42.122367 3036 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 22:42:42.122387 3036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:42.123891 3036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab
--principal=oryx/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.197.126:35613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:42.124128 3036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:42.124380 3036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:42.130709 3043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:42.130694 3041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:42.130676 3046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:42.130959 3036 server_base.cc:1034] running on GCE node
I20260805 22:42:42.131206 3036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:42.131507 3036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:42.132675 3036 hybrid_clock.cc:648] HybridClock initialized: now 1785969762132655 us; error 39 us; skew 500 ppm
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969762, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:42.135455 3036 init.cc:377] Logged in from keytab as oryx/127.23.197.65@KRBTEST.COM (short username oryx)
I20260805 22:42:42.135952 3036 webserver.cc:460] Webserver started at http://127.23.197.65:35071/ using document root <none> and password file <none>
I20260805 22:42:42.136243 3036 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:42.136296 3036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:42.136410 3036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:42.137595 3036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "afa13bd8003540aeabc0ed49805b1610"
format_stamp: "Formatted at 2026-08-05 22:42:42 on dist-test-slave-g170"
server_key: "9c0648d8ec553365637551e61ba74046"
server_key_iv: "7b6073b6b489b79e0d4715c18cb95f56"
server_key_version: "encryptionkey@0"
I20260805 22:42:42.137931 3036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "afa13bd8003540aeabc0ed49805b1610"
format_stamp: "Formatted at 2026-08-05 22:42:42 on dist-test-slave-g170"
server_key: "9c0648d8ec553365637551e61ba74046"
server_key_iv: "7b6073b6b489b79e0d4715c18cb95f56"
server_key_version: "encryptionkey@0"
I20260805 22:42:42.139482 3036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260805 22:42:42.140276 3057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:42.140476 3036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:42.140601 3036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "afa13bd8003540aeabc0ed49805b1610"
format_stamp: "Formatted at 2026-08-05 22:42:42 on dist-test-slave-g170"
server_key: "9c0648d8ec553365637551e61ba74046"
server_key_iv: "7b6073b6b489b79e0d4715c18cb95f56"
server_key_version: "encryptionkey@0"
I20260805 22:42:42.140682 3036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:42.158820 3036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:42.161098 3036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:42.161302 3036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:42.161947 3036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:42.162010 3036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:42.162060 3036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:42.162094 3036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969762, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:42.169706 3036 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:39507
I20260805 22:42:42.170120 3036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:42.170282 3210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:39507 every 8 connection(s)
I20260805 22:42:42.172005 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 3036
I20260805 22:42:42.172156 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:42.172422 24341 external_mini_cluster.cc:1428] Setting key b62c62f2c67f194f495f7bcc318d6a6c
I20260805 22:42:42.173560 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.162396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:35613 (local address 127.23.197.65:37093)
0805 22:42:42.162551 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.162593 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.163188 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.163468 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.163471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.163881 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:42.164188 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.164194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.165393 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.165395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.166771 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.166775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.167375 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.167377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.167580 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.168198 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:42.168208 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:42.170363 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:42.171913 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.171921 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.171923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.172191 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.172444 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.172444 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.172445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.172587 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.172945 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:42.172952 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:42.173112 (+ 160us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.173232 (+ 120us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.173400 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260805 22:42:42.173552 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.162419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:37093 (local address 127.23.197.126:35613)
0805 22:42:42.162498 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.162501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.162692 (+ 191us) server_negotiation.cc:407] Connection header received
0805 22:42:42.163298 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.163300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.163315 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.163407 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:42.164306 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.165286 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.166879 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.167283 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.167369 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.170474 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:42.170476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:42.170476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:42.170486 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:42.171810 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.172273 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.172275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.172276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.172359 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.172671 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.172672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.172672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.172869 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:42.172949 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.173284 (+ 335us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.173413 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 22:42:42.174002 3212 heartbeater.cc:347] Connected to a master server at 127.23.197.126:35613
I20260805 22:42:42.174081 3212 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:42.174273 3212 heartbeater.cc:511] Master 127.23.197.126:35613 requested a full tablet report, sending...
I20260805 22:42:42.174829 2936 ts_manager.cc:194] Registered new tserver with Master: afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507)
I20260805 22:42:42.175765 2936 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:37093
I20260805 22:42:42.182021 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.175223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:44333 (local address 127.23.197.126:35613)
0805 22:42:42.175258 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.175259 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.175362 (+ 103us) server_negotiation.cc:407] Connection header received
0805 22:42:42.175444 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.175445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.175454 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.175480 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:42.176072 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.176909 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.177962 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.178162 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.178239 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.179282 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:42.179284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:42.179284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:42.179290 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:42.180747 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.181177 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.181178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.181179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.181238 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.181453 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.181454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.181454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.181555 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:42.181610 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.181885 (+ 275us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.181956 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260805 22:42:42.182097 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.175074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:35613 (local address 127.23.197.65:44333)
0805 22:42:42.175309 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.175318 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.175383 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.175533 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.175534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.175711 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:42.175796 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.175799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.177007 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.177009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.177877 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.177880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.178240 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.178241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.178314 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.178752 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:42.178758 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:42.179197 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:42.180874 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.180876 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.180876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.181105 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.181307 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.181308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.181309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.181388 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.181659 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:42.181660 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:42.181774 (+ 114us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.181862 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.181969 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab.
Entry for principal oryx/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab.
I20260805 22:42:42.203498 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:35613
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab
--principal=oryx/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:42.300521 3226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:42.300822 3226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:42.300870 3226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:42.302234 3226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:42.302302 3226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:42.302340 3226 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 22:42:42.302382 3226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:42.304160 3226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab
--principal=oryx/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.197.126:35613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:42.304474 3226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:42.304747 3226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:42.310480 3233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:42.310499 3236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:42.310690 3226 server_base.cc:1034] running on GCE node
W20260805 22:42:42.310492 3232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:42.311008 3226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:42.311244 3226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:42.312407 3226 hybrid_clock.cc:648] HybridClock initialized: now 1785969762312395 us; error 31 us; skew 500 ppm
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969762, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:42.315344 3226 init.cc:377] Logged in from keytab as oryx/127.23.197.66@KRBTEST.COM (short username oryx)
I20260805 22:42:42.315907 3226 webserver.cc:460] Webserver started at http://127.23.197.66:44415/ using document root <none> and password file <none>
I20260805 22:42:42.316183 3226 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:42.316241 3226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:42.316392 3226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:42.317579 3226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "9818810f23fa4edc8a81dcb8e8df37ca"
format_stamp: "Formatted at 2026-08-05 22:42:42 on dist-test-slave-g170"
server_key: "7aafc49d85b28b17609df258709f9c0e"
server_key_iv: "f64090b02525830532e7cae4e13e0eac"
server_key_version: "encryptionkey@0"
I20260805 22:42:42.317950 3226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "9818810f23fa4edc8a81dcb8e8df37ca"
format_stamp: "Formatted at 2026-08-05 22:42:42 on dist-test-slave-g170"
server_key: "7aafc49d85b28b17609df258709f9c0e"
server_key_iv: "f64090b02525830532e7cae4e13e0eac"
server_key_version: "encryptionkey@0"
I20260805 22:42:42.319378 3226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:42:42.320147 3248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:42.320430 3226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:42.320536 3226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "9818810f23fa4edc8a81dcb8e8df37ca"
format_stamp: "Formatted at 2026-08-05 22:42:42 on dist-test-slave-g170"
server_key: "7aafc49d85b28b17609df258709f9c0e"
server_key_iv: "f64090b02525830532e7cae4e13e0eac"
server_key_version: "encryptionkey@0"
I20260805 22:42:42.320613 3226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:42.331004 3226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:42.333316 3226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:42.333472 3226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:42.334070 3226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:42.334129 3226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:42.334163 3226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:42.334187 3226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969762, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:42.341763 3226 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:37765
I20260805 22:42:42.341847 3417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:37765 every 8 connection(s)
I20260805 22:42:42.342180 3226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:42:42.344431 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.334540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:43037 (local address 127.23.197.126:35613)
0805 22:42:42.334610 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.334613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.334876 (+ 263us) server_negotiation.cc:407] Connection header received
0805 22:42:42.335500 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.335502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.335517 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.335576 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:42.336352 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.337061 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.338109 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.338338 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.338416 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.341456 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:42.341457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:42.341458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:42.341467 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:42.343083 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.343507 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.343508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.343508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.343565 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.343830 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.343831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.343831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.343945 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:42.344003 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.344299 (+ 296us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.344368 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 22:42:42.344702 3259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.334571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:35613 (local address 127.23.197.66:43037)
0805 22:42:42.334733 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.334775 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.335327 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.335656 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.335659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.336014 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:42.336248 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.336254 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.337160 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.337161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.338013 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.338015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.338425 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.338426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.338563 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.339162 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:42.339171 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:42.341346 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:42.343177 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.343191 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.343193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.343431 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.343640 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.343640 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.343641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.343769 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.344036 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:42.344042 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:42.344175 (+ 133us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.344276 (+ 101us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.344415 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260805 22:42:42.345247 3418 heartbeater.cc:347] Connected to a master server at 127.23.197.126:35613
I20260805 22:42:42.345331 3418 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:42.345534 3418 heartbeater.cc:511] Master 127.23.197.126:35613 requested a full tablet report, sending...
I20260805 22:42:42.345966 2936 ts_manager.cc:194] Registered new tserver with Master: 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765)
I20260805 22:42:42.346485 2936 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:43037
I20260805 22:42:42.349675 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 3226
I20260805 22:42:42.349783 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:42.350025 24341 external_mini_cluster.cc:1428] Setting key 5085eeb7af98a13d4ab7d8725ab5b624
I20260805 22:42:42.352700 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.346276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:35613 (local address 127.23.197.66:56031)
0805 22:42:42.346408 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.346419 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.346505 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.346650 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.346651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.346835 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:42.346900 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.346902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.347594 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.347595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.348133 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.348135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.348518 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.348519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.348588 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.349178 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:42.349183 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:42.349509 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:42.351570 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.351571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.351571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.351790 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.351993 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.351994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.351994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.352051 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.352399 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:42.352400 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:42.352440 (+ 40us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.352507 (+ 67us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.352595 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260805 22:42:42.352823 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.346341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:56031 (local address 127.23.197.126:35613)
0805 22:42:42.346385 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.346387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.346488 (+ 101us) server_negotiation.cc:407] Connection header received
0805 22:42:42.346580 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.346581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.346591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.346636 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:42.346979 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.347496 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.348223 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.348433 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.348515 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.349632 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:42.349634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:42.349634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:42.349641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:42.351476 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.351859 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.351861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.351861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.351927 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.352117 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.352118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.352118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.352331 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:42.352403 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.352553 (+ 150us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.352644 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for oryx/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab.
Entry for principal oryx/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab.
I20260805 22:42:42.380038 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:35613
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab
--principal=oryx/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:42.467018 3428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:42.467260 3428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:42.467289 3428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:42.468665 3428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:42.468713 3428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:42.468739 3428 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 22:42:42.468814 3428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:42.470525 3428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab
--principal=oryx/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.197.126:35613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:42.470788 3428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:42.471046 3428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:42.476734 3435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:42.476825 3434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:42.476717 3438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:42.477005 3428 server_base.cc:1034] running on GCE node
I20260805 22:42:42.477232 3428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:42.477487 3428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:42.478701 3428 hybrid_clock.cc:648] HybridClock initialized: now 1785969762478686 us; error 44 us; skew 500 ppm
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969762, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:42.481415 3428 init.cc:377] Logged in from keytab as oryx/127.23.197.67@KRBTEST.COM (short username oryx)
I20260805 22:42:42.481902 3428 webserver.cc:460] Webserver started at http://127.23.197.67:40985/ using document root <none> and password file <none>
I20260805 22:42:42.482138 3428 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:42.482204 3428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:42.482347 3428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:42.483661 3428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "482b2f2bcc874a249c7a645859e69dcd"
format_stamp: "Formatted at 2026-08-05 22:42:42 on dist-test-slave-g170"
server_key: "dd46b1c7baa74e13f55e7fbeaf7237dd"
server_key_iv: "a50349cbd31abff61f3e3d85e34b24f8"
server_key_version: "encryptionkey@0"
I20260805 22:42:42.484050 3428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "482b2f2bcc874a249c7a645859e69dcd"
format_stamp: "Formatted at 2026-08-05 22:42:42 on dist-test-slave-g170"
server_key: "dd46b1c7baa74e13f55e7fbeaf7237dd"
server_key_iv: "a50349cbd31abff61f3e3d85e34b24f8"
server_key_version: "encryptionkey@0"
I20260805 22:42:42.485541 3428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:42.486395 3447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:42.486598 3428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:42.486708 3428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "482b2f2bcc874a249c7a645859e69dcd"
format_stamp: "Formatted at 2026-08-05 22:42:42 on dist-test-slave-g170"
server_key: "dd46b1c7baa74e13f55e7fbeaf7237dd"
server_key_iv: "a50349cbd31abff61f3e3d85e34b24f8"
server_key_version: "encryptionkey@0"
I20260805 22:42:42.486783 3428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:42.504824 3428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:42.507130 3428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:42.507290 3428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:42.507871 3428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:42.507928 3428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:42.507973 3428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:42.508013 3428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969762, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:42.515921 3428 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:44609
I20260805 22:42:42.516078 3617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:44609 every 8 connection(s)
I20260805 22:42:42.516422 3428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:42.518514 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.508287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:35303 (local address 127.23.197.126:35613)
0805 22:42:42.508354 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.508357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.508603 (+ 246us) server_negotiation.cc:407] Connection header received
0805 22:42:42.509151 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.509153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.509179 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.509270 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:42.509844 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.510708 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.511745 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.512088 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.512188 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.515371 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:42.515373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:42.515373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:42.515382 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:42.517089 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.517508 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.517509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.517510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.517576 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.517892 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.517893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.517893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.518016 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:42.518078 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.518377 (+ 299us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.518449 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 22:42:42.518770 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.508296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:35613 (local address 127.23.197.67:35303)
0805 22:42:42.508463 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.508513 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.509054 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.509321 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.509325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.509554 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:42.509750 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.509754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.510817 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.510818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.511644 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.511646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.512192 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.512193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.512336 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.512930 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:42.512942 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:42.515270 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:42.517168 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.517173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.517175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.517411 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.517684 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.517685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.517686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.517809 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.518101 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:42.518107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:42.518237 (+ 130us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.518343 (+ 106us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.518475 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260805 22:42:42.519310 3619 heartbeater.cc:347] Connected to a master server at 127.23.197.126:35613
I20260805 22:42:42.519393 3619 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:42.519584 3619 heartbeater.cc:511] Master 127.23.197.126:35613 requested a full tablet report, sending...
I20260805 22:42:42.520076 2937 ts_manager.cc:194] Registered new tserver with Master: 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609)
I20260805 22:42:42.520572 2937 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:35303
I20260805 22:42:42.526211 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 3428
I20260805 22:42:42.526322 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:42.526546 24341 external_mini_cluster.cc:1428] Setting key f76c9bed908d6439df74559485581df7
I20260805 22:42:42.526686 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.520500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:35613 (local address 127.23.197.67:47181)
0805 22:42:42.520637 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.520649 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.520703 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.520949 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.520950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.521171 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:42.521257 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.521259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.522129 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.522130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.522965 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.522968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.523361 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.523362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.523412 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.523798 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:42.523802 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:42.524086 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:42.525632 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.525633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.525633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.525805 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.525984 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:42.525984 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:42.525985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:42.526038 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:42.526339 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:42.526340 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:42.526392 (+ 52us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.526473 (+ 81us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.526553 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260805 22:42:42.526748 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.520718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:47181 (local address 127.23.197.126:35613)
0805 22:42:42.520805 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.520807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.520814 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:42:42.520834 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.520835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.520846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.520902 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:42.521375 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.522022 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.523043 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.523285 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.523354 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.524184 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:42.524185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:42.524186 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:42.524192 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:42.525564 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.525876 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.525877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.525877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.525928 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.526116 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.526117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.526118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.526256 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:42.526337 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.526515 (+ 178us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.526593 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260805 22:42:42.527688 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969762, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
W20260805 22:42:42.550777 3007 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:42.544746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48146 (local address 127.23.197.126:35613)
0805 22:42:42.544839 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.544842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.544854 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:42:42.544877 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.544878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.544887 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.544948 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:42.545686 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.546209 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.546986 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.547184 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.547258 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.550598 (+ 3340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48146: BlockingRecv error: recv got EOF from 127.0.0.1:48146 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 05 22:42:42 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969762, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:42.558727 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.551712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.23.197.126:35613)
0805 22:42:42.551794 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.551797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.551886 (+ 89us) server_negotiation.cc:407] Connection header received
0805 22:42:42.551990 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.551991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.552002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.552061 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:42.552557 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.553101 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.553846 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.554039 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.554109 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.556287 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:42.556289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:42.556289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:42.556297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:42.557606 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.557938 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.557940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.557940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.557999 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:42.558194 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:42.558195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:42.558196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:42.558308 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:42.558431 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.558542 (+ 111us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.558624 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 22:42:42.560431 2937 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:42.562109 2937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:42.568747 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.564746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.23.197.67:44609)
0805 22:42:42.564971 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.564977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.564989 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:42:42.565036 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.565038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.565058 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.565356 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:42.565417 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.566508 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.567791 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.568333 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.568426 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.568510 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.568613 (+ 103us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.568654 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":118,"threads_started":1}
I20260805 22:42:42.568742 3652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.564525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:44609 (local address 127.0.0.1:60126)
0805 22:42:42.564700 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.564741 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.564858 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.565189 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.565193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.565195 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:42.565267 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.565269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.566595 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.566595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.567692 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.567694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.568425 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.568426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.568505 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.568520 (+ 15us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.568597 (+ 77us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.568653 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":31,"thread_start_us":50,"threads_started":1}
I20260805 22:42:42.568910 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.564531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:39507 (local address 127.0.0.1:52194)
0805 22:42:42.564972 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.564982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.565021 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.565263 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.565263 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.565265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:42.565319 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.565321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.566624 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.566625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.567558 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.567560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.568612 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.568613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.568696 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.568699 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.568747 (+ 48us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.568816 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":113,"threads_started":1}
I20260805 22:42:42.569195 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.564841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52194 (local address 127.23.197.65:39507)
0805 22:42:42.565031 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.565034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.565044 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:42.565084 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.565086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.565101 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.565187 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:42.565399 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.566536 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.567643 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.568402 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.568478 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.568558 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.569048 (+ 490us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.569093 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260805 22:42:42.569465 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.564535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37765 (local address 127.0.0.1:58106)
0805 22:42:42.565037 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.565047 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.565090 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.565463 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.565464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.565466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:42.565525 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.565526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.566374 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.566375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.567670 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.567673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.568626 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.568627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.568690 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.568694 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.569316 (+ 622us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.569368 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":98,"threads_started":1}
I20260805 22:42:42.569234 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.564741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58106 (local address 127.23.197.66:37765)
0805 22:42:42.565230 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.565235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.565245 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:42.565285 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.565287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.565315 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.565392 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:42.565602 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.566284 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.567767 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.568465 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.568524 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.568596 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.569076 (+ 480us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.569117 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":128,"threads_started":1}
I20260805 22:42:42.569790 3121 tablet_service.cc:1467] Processing CreateTablet for tablet 9b803c063d784d1dbd67ebeb1bb8bd8a (DEFAULT_TABLE table=test-table [id=4f9155c0e8234c74bd1c497f1d6b0a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:42.569777 3518 tablet_service.cc:1467] Processing CreateTablet for tablet 9b803c063d784d1dbd67ebeb1bb8bd8a (DEFAULT_TABLE table=test-table [id=4f9155c0e8234c74bd1c497f1d6b0a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:42.570199 3121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b803c063d784d1dbd67ebeb1bb8bd8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:42.570199 3518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b803c063d784d1dbd67ebeb1bb8bd8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:42.570286 3328 tablet_service.cc:1467] Processing CreateTablet for tablet 9b803c063d784d1dbd67ebeb1bb8bd8a (DEFAULT_TABLE table=test-table [id=4f9155c0e8234c74bd1c497f1d6b0a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:42.570621 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b803c063d784d1dbd67ebeb1bb8bd8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:42.573957 3656 tablet_bootstrap.cc:492] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: Bootstrap starting.
I20260805 22:42:42.574352 3657 tablet_bootstrap.cc:492] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: Bootstrap starting.
I20260805 22:42:42.574734 3656 tablet_bootstrap.cc:654] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:42.574710 3659 tablet_bootstrap.cc:492] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: Bootstrap starting.
I20260805 22:42:42.574997 3657 tablet_bootstrap.cc:654] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:42.575181 3656 log.cc:824] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:42.575371 3657 log.cc:824] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:42.575434 3659 tablet_bootstrap.cc:654] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:42.575788 3659 log.cc:824] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:42.576027 3656 tablet_bootstrap.cc:492] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: No bootstrap required, opened a new log
I20260805 22:42:42.576110 3656 ts_tablet_manager.cc:1397] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:42.576840 3659 tablet_bootstrap.cc:492] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: No bootstrap required, opened a new log
I20260805 22:42:42.576929 3659 ts_tablet_manager.cc:1397] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:42.577138 3657 tablet_bootstrap.cc:492] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: No bootstrap required, opened a new log
I20260805 22:42:42.577212 3657 ts_tablet_manager.cc:1397] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260805 22:42:42.578246 3656 raft_consensus.cc:348] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } }
I20260805 22:42:42.578438 3656 raft_consensus.cc:374] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:42.578482 3656 raft_consensus.cc:729] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9818810f23fa4edc8a81dcb8e8df37ca, State: Initialized, Role: FOLLOWER
I20260805 22:42:42.578382 3659 raft_consensus.cc:348] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } }
I20260805 22:42:42.578536 3659 raft_consensus.cc:374] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:42.578580 3659 raft_consensus.cc:729] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa13bd8003540aeabc0ed49805b1610, State: Initialized, Role: FOLLOWER
I20260805 22:42:42.578629 3656 consensus_queue.cc:260] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } }
I20260805 22:42:42.578701 3659 consensus_queue.cc:260] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } }
I20260805 22:42:42.578927 3656 ts_tablet_manager.cc:1428] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:42:42.578948 3418 heartbeater.cc:503] Master 127.23.197.126:35613 was elected leader, sending a full tablet report...
I20260805 22:42:42.578967 3659 ts_tablet_manager.cc:1428] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:42:42.579104 3212 heartbeater.cc:503] Master 127.23.197.126:35613 was elected leader, sending a full tablet report...
I20260805 22:42:42.579272 3657 raft_consensus.cc:348] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } }
I20260805 22:42:42.579416 3657 raft_consensus.cc:374] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:42.579439 3657 raft_consensus.cc:729] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b2f2bcc874a249c7a645859e69dcd, State: Initialized, Role: FOLLOWER
I20260805 22:42:42.579536 3657 consensus_queue.cc:260] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } }
I20260805 22:42:42.579787 3657 ts_tablet_manager.cc:1428] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:42:42.579862 3619 heartbeater.cc:503] Master 127.23.197.126:35613 was elected leader, sending a full tablet report...
W20260805 22:42:42.593008 3419 tablet.cc:2369] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:42:42.671314 3213 tablet.cc:2369] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:42:42.767120 3623 tablet.cc:2369] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:42:42.791491 3668 raft_consensus.cc:482] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:42.791667 3668 raft_consensus.cc:504] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } }
I20260805 22:42:42.792074 3668 leader_election.cc:288] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609), 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765)
I20260805 22:42:42.795387 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.792498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:34341 (local address 127.23.197.67:44609)
0805 22:42:42.792584 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.792651 (+ 67us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.792708 (+ 57us) server_negotiation.cc:407] Connection header received
0805 22:42:42.792825 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.792826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.792835 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.792887 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:42.793110 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.793598 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.794574 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.794958 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.795019 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.795027 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.795245 (+ 218us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.795295 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 22:42:42.795410 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.792402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:48161 (local address 127.23.197.66:37765)
0805 22:42:42.792463 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.792466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.792572 (+ 106us) server_negotiation.cc:407] Connection header received
0805 22:42:42.792714 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.792716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.792727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.792823 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:42.793009 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.793521 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.794562 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.794966 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.795103 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.795112 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.795292 (+ 180us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.795336 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260805 22:42:42.795424 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.792307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:44609 (local address 127.23.197.65:34341)
0805 22:42:42.792626 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.792652 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.792722 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.792904 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.792905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.792907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:42.792999 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.793001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.793691 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.793692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.794459 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.794461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.795074 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.795076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.795159 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.795164 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.795218 (+ 54us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.795275 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":1,"thread_start_us":87,"threads_started":1}
I20260805 22:42:42.795444 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.792288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37765 (local address 127.23.197.65:48161)
0805 22:42:42.792479 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.792491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.792582 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.792835 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.792837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.792840 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:42.792931 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.792933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.793617 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.793618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.794457 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.794460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.795118 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.795118 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.795174 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.795178 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.795273 (+ 95us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.795317 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":111,"threads_started":1}
I20260805 22:42:42.795897 3352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b803c063d784d1dbd67ebeb1bb8bd8a" candidate_uuid: "afa13bd8003540aeabc0ed49805b1610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" is_pre_election: true
I20260805 22:42:42.795948 3552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b803c063d784d1dbd67ebeb1bb8bd8a" candidate_uuid: "afa13bd8003540aeabc0ed49805b1610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b2f2bcc874a249c7a645859e69dcd" is_pre_election: true
I20260805 22:42:42.796052 3352 raft_consensus.cc:2454] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa13bd8003540aeabc0ed49805b1610 in term 0.
I20260805 22:42:42.796144 3552 raft_consensus.cc:2454] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa13bd8003540aeabc0ed49805b1610 in term 0.
I20260805 22:42:42.796320 3061 leader_election.cc:302] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9818810f23fa4edc8a81dcb8e8df37ca, afa13bd8003540aeabc0ed49805b1610; no voters:
I20260805 22:42:42.796454 3668 raft_consensus.cc:2781] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:42.796517 3668 raft_consensus.cc:482] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:42.796541 3668 raft_consensus.cc:3037] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:42.797300 3668 raft_consensus.cc:504] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } }
I20260805 22:42:42.797446 3668 leader_election.cc:288] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [CANDIDATE]: Term 1 election: Requested vote from peers 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609), 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765)
I20260805 22:42:42.797619 3552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b803c063d784d1dbd67ebeb1bb8bd8a" candidate_uuid: "afa13bd8003540aeabc0ed49805b1610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b2f2bcc874a249c7a645859e69dcd"
I20260805 22:42:42.797660 3352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b803c063d784d1dbd67ebeb1bb8bd8a" candidate_uuid: "afa13bd8003540aeabc0ed49805b1610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9818810f23fa4edc8a81dcb8e8df37ca"
I20260805 22:42:42.797722 3552 raft_consensus.cc:3037] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:42.797722 3352 raft_consensus.cc:3037] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:42.798439 3352 raft_consensus.cc:2454] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa13bd8003540aeabc0ed49805b1610 in term 1.
I20260805 22:42:42.798532 3552 raft_consensus.cc:2454] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa13bd8003540aeabc0ed49805b1610 in term 1.
I20260805 22:42:42.798609 3061 leader_election.cc:302] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9818810f23fa4edc8a81dcb8e8df37ca, afa13bd8003540aeabc0ed49805b1610; no voters:
I20260805 22:42:42.798738 3668 raft_consensus.cc:2781] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:42.798888 3668 raft_consensus.cc:686] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 1 LEADER]: Becoming Leader. State: Replica: afa13bd8003540aeabc0ed49805b1610, State: Running, Role: LEADER
I20260805 22:42:42.799000 3668 consensus_queue.cc:237] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } }
I20260805 22:42:42.799871 2936 catalog_manager.cc:5495] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 reported cstate change: term changed from 0 to 1, leader changed from <none> to afa13bd8003540aeabc0ed49805b1610 (127.23.197.65). New cstate: current_term: 1 leader_uuid: "afa13bd8003540aeabc0ed49805b1610" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } health_report { overall_health: HEALTHY } } }
I20260805 22:42:42.852176 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.849792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52196 (local address 127.23.197.65:39507)
0805 22:42:42.849857 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.849858 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.849866 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:42.849904 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.849905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.849924 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.849979 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:42.850162 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.850627 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.851467 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.851665 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.851728 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.851912 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:42.852015 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:42.852077 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.852091 (+ 14us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.852108 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 22:42:42.854138 3552 raft_consensus.cc:1261] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Refusing update from remote peer afa13bd8003540aeabc0ed49805b1610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:42.854138 3352 raft_consensus.cc:1261] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Refusing update from remote peer afa13bd8003540aeabc0ed49805b1610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:42.854642 3673 consensus_queue.cc:1035] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:42.854797 3668 consensus_queue.cc:1035] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:42.857964 3678 mvcc.cc:205] Tried to move back new op lower bound from 7315332148647575552 to 7315332148425318400. Current Snapshot: MvccSnapshot[applied={T|T < 7315332148647575552}]
I20260805 22:42:42.859669 2936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:42.859773 2936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:42.860531 2936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728: Sending DeleteTablet for 3 replicas of tablet 9b803c063d784d1dbd67ebeb1bb8bd8a
I20260805 22:42:42.860898 3518 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b803c063d784d1dbd67ebeb1bb8bd8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:42 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:60126
I20260805 22:42:42.861122 3121 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b803c063d784d1dbd67ebeb1bb8bd8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:42 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:52194
I20260805 22:42:42.861155 3328 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b803c063d784d1dbd67ebeb1bb8bd8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:42 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:58106
I20260805 22:42:42.861344 3694 tablet_replica.cc:330] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: stopping tablet replica
I20260805 22:42:42.861477 3694 raft_consensus.cc:2229] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:42.861451 2936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:42.861584 2936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:42.861613 3694 raft_consensus.cc:2258] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:42.862061 3694 ts_tablet_manager.cc:1905] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:42.862854 3328 tablet_service.cc:1467] Processing CreateTablet for tablet 24652459f3f34b979f60d6f1fc6853f7 (DEFAULT_TABLE table=test-table [id=d189f31ec57647e6a39d20d346f4eb88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:42.862874 3121 tablet_service.cc:1467] Processing CreateTablet for tablet 24652459f3f34b979f60d6f1fc6853f7 (DEFAULT_TABLE table=test-table [id=d189f31ec57647e6a39d20d346f4eb88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:42.862994 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24652459f3f34b979f60d6f1fc6853f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:42.863003 3121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24652459f3f34b979f60d6f1fc6853f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:42.863926 3656 tablet_bootstrap.cc:492] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: Bootstrap starting.
I20260805 22:42:42.864424 3656 tablet_bootstrap.cc:654] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:42.864430 3518 tablet_service.cc:1467] Processing CreateTablet for tablet 24652459f3f34b979f60d6f1fc6853f7 (DEFAULT_TABLE table=test-table [id=d189f31ec57647e6a39d20d346f4eb88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:42.864550 3518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24652459f3f34b979f60d6f1fc6853f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:42.864564 3692 tablet_replica.cc:330] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: stopping tablet replica
I20260805 22:42:42.864670 3692 raft_consensus.cc:2229] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:42.864800 3692 raft_consensus.cc:2258] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:42.865293 3692 ts_tablet_manager.cc:1905] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:42.865769 3657 tablet_bootstrap.cc:492] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: Bootstrap starting.
I20260805 22:42:42.865797 3659 tablet_bootstrap.cc:492] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: Bootstrap starting.
I20260805 22:42:42.866183 3659 tablet_bootstrap.cc:654] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:42.866230 3657 tablet_bootstrap.cc:654] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:42.866937 3659 tablet_bootstrap.cc:492] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: No bootstrap required, opened a new log
I20260805 22:42:42.866990 3659 ts_tablet_manager.cc:1397] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:42.867141 3659 raft_consensus.cc:348] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:42.867251 3659 raft_consensus.cc:374] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:42.867275 3659 raft_consensus.cc:729] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa13bd8003540aeabc0ed49805b1610, State: Initialized, Role: FOLLOWER
I20260805 22:42:42.867337 3659 consensus_queue.cc:260] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:42.867442 3659 ts_tablet_manager.cc:1428] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:42.867879 3694 ts_tablet_manager.cc:1918] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:42.867960 3694 log.cc:1196] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/9b803c063d784d1dbd67ebeb1bb8bd8a
I20260805 22:42:42.868019 3692 ts_tablet_manager.cc:1918] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:42.868101 3692 log.cc:1196] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/9b803c063d784d1dbd67ebeb1bb8bd8a
I20260805 22:42:42.868374 3692 ts_tablet_manager.cc:1939] T 9b803c063d784d1dbd67ebeb1bb8bd8a P afa13bd8003540aeabc0ed49805b1610: Deleting consensus metadata
I20260805 22:42:42.868561 3694 ts_tablet_manager.cc:1939] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting consensus metadata
I20260805 22:42:42.868873 3691 tablet_replica.cc:330] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: stopping tablet replica
I20260805 22:42:42.869030 2917 catalog_manager.cc:4841] TS 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765): tablet 9b803c063d784d1dbd67ebeb1bb8bd8a (table test-table [id=4f9155c0e8234c74bd1c497f1d6b0a7c]) successfully deleted
I20260805 22:42:42.869306 3657 tablet_bootstrap.cc:492] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: No bootstrap required, opened a new log
I20260805 22:42:42.869356 3657 ts_tablet_manager.cc:1397] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260805 22:42:42.869478 3656 tablet_bootstrap.cc:492] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: No bootstrap required, opened a new log
I20260805 22:42:42.869513 3656 ts_tablet_manager.cc:1397] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260805 22:42:42.869511 3657 raft_consensus.cc:348] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:42.869580 3657 raft_consensus.cc:374] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:42.869601 3657 raft_consensus.cc:729] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b2f2bcc874a249c7a645859e69dcd, State: Initialized, Role: FOLLOWER
I20260805 22:42:42.869656 3657 consensus_queue.cc:260] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:42.869627 3656 raft_consensus.cc:348] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:42.869714 3656 raft_consensus.cc:374] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:42.869737 3656 raft_consensus.cc:729] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9818810f23fa4edc8a81dcb8e8df37ca, State: Initialized, Role: FOLLOWER
I20260805 22:42:42.869750 3657 ts_tablet_manager.cc:1428] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:42.869781 3656 consensus_queue.cc:260] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:42.869902 3656 ts_tablet_manager.cc:1428] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:42.870266 3691 raft_consensus.cc:2229] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:42.870419 3691 raft_consensus.cc:2258] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:42.870899 2922 catalog_manager.cc:4841] TS afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507): tablet 9b803c063d784d1dbd67ebeb1bb8bd8a (table test-table [id=4f9155c0e8234c74bd1c497f1d6b0a7c]) successfully deleted
I20260805 22:42:42.871372 3691 ts_tablet_manager.cc:1905] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:42.872925 3691 ts_tablet_manager.cc:1918] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:42.873001 3691 log.cc:1196] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/9b803c063d784d1dbd67ebeb1bb8bd8a
I20260805 22:42:42.873274 3691 ts_tablet_manager.cc:1939] T 9b803c063d784d1dbd67ebeb1bb8bd8a P 482b2f2bcc874a249c7a645859e69dcd: Deleting consensus metadata
I20260805 22:42:42.873696 2920 catalog_manager.cc:4841] TS 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609): tablet 9b803c063d784d1dbd67ebeb1bb8bd8a (table test-table [id=4f9155c0e8234c74bd1c497f1d6b0a7c]) successfully deleted
I20260805 22:42:42.969570 3669 raft_consensus.cc:482] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:42.969745 3669 raft_consensus.cc:504] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:42.970129 3669 leader_election.cc:288] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507), 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765)
I20260805 22:42:42.973878 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.970339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:39507 (local address 127.23.197.67:35167)
0805 22:42:42.970405 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.970418 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.970473 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.970678 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.970679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.970681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:42.970757 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.970759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.971586 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.971587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.972854 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.972859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.973605 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.973606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.973704 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.973708 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.973756 (+ 48us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.973801 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260805 22:42:42.974294 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.970395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:35167 (local address 127.23.197.65:39507)
0805 22:42:42.970445 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.970447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.970459 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:42:42.970557 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.970558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.970571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.970623 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:42.970819 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.971478 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.972959 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.973485 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.973559 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.973569 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.974167 (+ 598us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.974214 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:42:42.974429 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.970436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:59877 (local address 127.23.197.66:37765)
0805 22:42:42.970484 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:42.970486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:42.970708 (+ 222us) server_negotiation.cc:407] Connection header received
0805 22:42:42.970772 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:42.970774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:42.970784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:42.970847 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:42.971143 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.971865 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.973394 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.973971 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.974024 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.974033 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:42.974284 (+ 251us) server_negotiation.cc:299] Negotiation successful
0805 22:42:42.974327 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260805 22:42:42.974505 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24652459f3f34b979f60d6f1fc6853f7" candidate_uuid: "482b2f2bcc874a249c7a645859e69dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa13bd8003540aeabc0ed49805b1610" is_pre_election: true
I20260805 22:42:42.974651 3150 raft_consensus.cc:2454] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482b2f2bcc874a249c7a645859e69dcd in term 0.
I20260805 22:42:42.974686 3700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:42.970341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37765 (local address 127.23.197.67:59877)
0805 22:42:42.970625 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:42.970636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:42.970703 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:42.970921 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:42.970923 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:42.970925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:42.971030 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.971032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.971971 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.971972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.973240 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:42.973243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:42.974111 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:42.974112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:42.974204 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:42.974207 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:42.974556 (+ 349us) client_negotiation.cc:240] Negotiation successful
0805 22:42:42.974598 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":155,"thread_start_us":81,"threads_started":1}
I20260805 22:42:42.974864 3452 leader_election.cc:302] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b2f2bcc874a249c7a645859e69dcd, afa13bd8003540aeabc0ed49805b1610; no voters:
I20260805 22:42:42.974910 3352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24652459f3f34b979f60d6f1fc6853f7" candidate_uuid: "482b2f2bcc874a249c7a645859e69dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" is_pre_election: true
I20260805 22:42:42.974995 3669 raft_consensus.cc:2781] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:42.974998 3352 raft_consensus.cc:2454] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482b2f2bcc874a249c7a645859e69dcd in term 0.
I20260805 22:42:42.975035 3669 raft_consensus.cc:482] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:42.975066 3669 raft_consensus.cc:3037] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:42.975767 3669 raft_consensus.cc:504] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:42.975925 3669 leader_election.cc:288] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [CANDIDATE]: Term 1 election: Requested vote from peers afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507), 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765)
I20260805 22:42:42.976081 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24652459f3f34b979f60d6f1fc6853f7" candidate_uuid: "482b2f2bcc874a249c7a645859e69dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa13bd8003540aeabc0ed49805b1610"
I20260805 22:42:42.976148 3150 raft_consensus.cc:3037] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:42.976148 3352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24652459f3f34b979f60d6f1fc6853f7" candidate_uuid: "482b2f2bcc874a249c7a645859e69dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9818810f23fa4edc8a81dcb8e8df37ca"
I20260805 22:42:42.976222 3352 raft_consensus.cc:3037] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:42.976891 3150 raft_consensus.cc:2454] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482b2f2bcc874a249c7a645859e69dcd in term 1.
I20260805 22:42:42.977017 3352 raft_consensus.cc:2454] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482b2f2bcc874a249c7a645859e69dcd in term 1.
I20260805 22:42:42.977082 3452 leader_election.cc:302] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b2f2bcc874a249c7a645859e69dcd, afa13bd8003540aeabc0ed49805b1610; no voters:
I20260805 22:42:42.977177 3669 raft_consensus.cc:2781] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:42.977309 3669 raft_consensus.cc:686] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 1 LEADER]: Becoming Leader. State: Replica: 482b2f2bcc874a249c7a645859e69dcd, State: Running, Role: LEADER
I20260805 22:42:42.977438 3669 consensus_queue.cc:237] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:42.978269 2937 catalog_manager.cc:5495] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd reported cstate change: term changed from 0 to 1, leader changed from <none> to 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67). New cstate: current_term: 1 leader_uuid: "482b2f2bcc874a249c7a645859e69dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } health_report { overall_health: HEALTHY } } }
Aug 05 22:42:43 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969761, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.126@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:43.012837 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.005636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.23.197.126:35613)
0805 22:42:43.005704 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.005706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.005832 (+ 126us) server_negotiation.cc:407] Connection header received
0805 22:42:43.005927 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.005928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.005943 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.006005 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:43.006480 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.006982 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.007710 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.007890 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.007983 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.010288 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:43.010290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:43.010290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:43.010298 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:43.011592 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:43.011990 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:43.011991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:43.011992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:43.012052 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:43.012265 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:43.012266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:43.012266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:43.012379 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:43.012451 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.012612 (+ 161us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.012699 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260805 22:42:43.012921 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.005565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:35613 (local address 127.0.0.1:48168)
0805 22:42:43.005746 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:43.005763 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:43.005852 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:43.006010 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:43.006010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:43.006347 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:43.006395 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.006397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.007071 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.007072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.007611 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.007612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.007982 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.007983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.008037 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.008603 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:43.008608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:43.010180 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:43.011692 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:43.011707 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:43.011710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:43.011914 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:43.012126 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:43.012127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:43.012127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:43.012189 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:43.012455 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:43.012464 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:43.012511 (+ 47us) client_negotiation.cc:769] Sending connection context
0805 22:42:43.012589 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 22:42:43.012797 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":77,"thread_start_us":93,"threads_started":1}
I20260805 22:42:43.015049 2937 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:48168:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260805 22:42:43.015285 2937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:43.016914 3328 tablet_service.cc:1467] Processing CreateTablet for tablet 64daef816b254891ae909841960aa294 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=abae39e7951d471681a579700e0c0fe6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 22:42:43.016930 3518 tablet_service.cc:1467] Processing CreateTablet for tablet 64daef816b254891ae909841960aa294 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=abae39e7951d471681a579700e0c0fe6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 22:42:43.017098 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64daef816b254891ae909841960aa294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:43.017098 3518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64daef816b254891ae909841960aa294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:43.018885 3656 tablet_bootstrap.cc:492] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: Bootstrap starting.
I20260805 22:42:43.018885 3657 tablet_bootstrap.cc:492] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: Bootstrap starting.
I20260805 22:42:43.018934 3121 tablet_service.cc:1467] Processing CreateTablet for tablet 64daef816b254891ae909841960aa294 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=abae39e7951d471681a579700e0c0fe6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 22:42:43.019059 3121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64daef816b254891ae909841960aa294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:43.019434 3657 tablet_bootstrap.cc:654] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:43.019434 3656 tablet_bootstrap.cc:654] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:43.020287 3657 tablet_bootstrap.cc:492] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: No bootstrap required, opened a new log
I20260805 22:42:43.020346 3657 ts_tablet_manager.cc:1397] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:43.020538 3659 tablet_bootstrap.cc:492] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: Bootstrap starting.
I20260805 22:42:43.020516 3657 raft_consensus.cc:348] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.020604 3657 raft_consensus.cc:374] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:43.020629 3657 raft_consensus.cc:729] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b2f2bcc874a249c7a645859e69dcd, State: Initialized, Role: FOLLOWER
I20260805 22:42:43.020685 3657 consensus_queue.cc:260] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.020813 3657 ts_tablet_manager.cc:1428] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:43.020864 3669 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.020913 3659 tablet_bootstrap.cc:654] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:43.020933 3669 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:43.021395 3656 tablet_bootstrap.cc:492] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: No bootstrap required, opened a new log
I20260805 22:42:43.021477 3656 ts_tablet_manager.cc:1397] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260805 22:42:43.021478 3659 tablet_bootstrap.cc:492] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: No bootstrap required, opened a new log
I20260805 22:42:43.021528 3659 ts_tablet_manager.cc:1397] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:43.021634 3659 raft_consensus.cc:348] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.021698 3659 raft_consensus.cc:374] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:43.021713 3659 raft_consensus.cc:729] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa13bd8003540aeabc0ed49805b1610, State: Initialized, Role: FOLLOWER
I20260805 22:42:43.021661 3656 raft_consensus.cc:348] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.021750 3656 raft_consensus.cc:374] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:43.021785 3656 raft_consensus.cc:729] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9818810f23fa4edc8a81dcb8e8df37ca, State: Initialized, Role: FOLLOWER
I20260805 22:42:43.021763 3659 consensus_queue.cc:260] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.021842 3656 consensus_queue.cc:260] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.021862 3659 ts_tablet_manager.cc:1428] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:43.021943 3656 ts_tablet_manager.cc:1428] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:43.021961 3685 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.022039 3685 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:43.022089 3666 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.022203 3666 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:43.035543 3666 raft_consensus.cc:482] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:43.035687 3666 raft_consensus.cc:504] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.036183 3666 leader_election.cc:288] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507), 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609)
I20260805 22:42:43.039274 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.036371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:44609 (local address 127.23.197.66:49495)
0805 22:42:43.036562 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:43.036574 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:43.036652 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:43.036876 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:43.036877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:43.036878 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:43.036968 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.036970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.037581 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.037582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.038430 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.038432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.038958 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.038959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.039044 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.039048 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:42:43.039124 (+ 76us) client_negotiation.cc:240] Negotiation successful
0805 22:42:43.039172 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260805 22:42:43.039261 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.036451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:49495 (local address 127.23.197.67:44609)
0805 22:42:43.036514 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.036517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.036641 (+ 124us) server_negotiation.cc:407] Connection header received
0805 22:42:43.036765 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.036766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.036810 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.036865 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:43.037044 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.037496 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.038521 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.038882 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.038957 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.038967 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.039139 (+ 172us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.039191 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260805 22:42:43.039322 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.036548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:35721 (local address 127.23.197.65:39507)
0805 22:42:43.036606 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.036607 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.036620 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:42:43.036696 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.036698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.036708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.036806 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:42:43.036977 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.037498 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.038528 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.038891 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.038972 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.038981 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.039191 (+ 210us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.039246 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260805 22:42:43.039618 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.036371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:39507 (local address 127.23.197.66:35721)
0805 22:42:43.036568 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:43.036580 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:43.036634 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:43.036817 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:43.036818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:43.036821 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:42:43.036908 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.036910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.037580 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.037581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.038428 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.038430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.038983 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.038984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.039082 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.039085 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:43.039490 (+ 405us) client_negotiation.cc:240] Negotiation successful
0805 22:42:43.039537 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":34,"thread_start_us":106,"threads_started":1}
I20260805 22:42:43.039608 3552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64daef816b254891ae909841960aa294" candidate_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b2f2bcc874a249c7a645859e69dcd" is_pre_election: true
I20260805 22:42:43.039703 3552 raft_consensus.cc:2454] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9818810f23fa4edc8a81dcb8e8df37ca in term 0.
I20260805 22:42:43.039867 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64daef816b254891ae909841960aa294" candidate_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa13bd8003540aeabc0ed49805b1610" is_pre_election: true
I20260805 22:42:43.039897 3252 leader_election.cc:302] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b2f2bcc874a249c7a645859e69dcd, 9818810f23fa4edc8a81dcb8e8df37ca; no voters:
I20260805 22:42:43.039973 3150 raft_consensus.cc:2454] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9818810f23fa4edc8a81dcb8e8df37ca in term 0.
I20260805 22:42:43.040027 3666 raft_consensus.cc:2781] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:43.040077 3666 raft_consensus.cc:482] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:43.040109 3666 raft_consensus.cc:3037] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:43.040884 3666 raft_consensus.cc:504] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.041019 3666 leader_election.cc:288] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [CANDIDATE]: Term 1 election: Requested vote from peers afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507), 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609)
I20260805 22:42:43.041157 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64daef816b254891ae909841960aa294" candidate_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa13bd8003540aeabc0ed49805b1610"
I20260805 22:42:43.041229 3150 raft_consensus.cc:3037] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:43.041183 3552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64daef816b254891ae909841960aa294" candidate_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b2f2bcc874a249c7a645859e69dcd"
I20260805 22:42:43.041287 3552 raft_consensus.cc:3037] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:43.041839 3150 raft_consensus.cc:2454] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9818810f23fa4edc8a81dcb8e8df37ca in term 1.
I20260805 22:42:43.041941 3552 raft_consensus.cc:2454] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9818810f23fa4edc8a81dcb8e8df37ca in term 1.
I20260805 22:42:43.042012 3253 leader_election.cc:302] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9818810f23fa4edc8a81dcb8e8df37ca, afa13bd8003540aeabc0ed49805b1610; no voters:
I20260805 22:42:43.042101 3666 raft_consensus.cc:2781] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:43.042263 3666 raft_consensus.cc:686] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 LEADER]: Becoming Leader. State: Replica: 9818810f23fa4edc8a81dcb8e8df37ca, State: Running, Role: LEADER
I20260805 22:42:43.042346 3666 consensus_queue.cc:237] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.042707 3716 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: TxnStatusTablet state changed. Reason: New leader 9818810f23fa4edc8a81dcb8e8df37ca. Latest consensus state: current_term: 1 leader_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.042865 3716 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: This TxnStatusTablet replica's current role is: LEADER
I20260805 22:42:43.043058 3719 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260805 22:42:43.043095 2936 catalog_manager.cc:5495] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66). New cstate: current_term: 1 leader_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:43.043191 3719 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260805 22:42:43.051306 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.048160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.23.197.66:37765)
0805 22:42:43.048203 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.048205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.048252 (+ 47us) server_negotiation.cc:407] Connection header received
0805 22:42:43.048359 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.048360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.048376 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.048436 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:43.048585 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.049136 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.050290 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.050575 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.050695 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.050909 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.051077 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.051145 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.051223 (+ 78us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.051249 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:42:43.051371 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.048113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37765 (local address 127.0.0.1:58108)
0805 22:42:43.048204 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:43.048216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:43.048271 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:43.048462 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:43.048463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:43.048465 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:43.048521 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.048523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.049234 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.049235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.050185 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.050189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.050696 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.050698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.050770 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.050789 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.051151 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.051160 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 22:42:43.051220 (+ 60us) client_negotiation.cc:240] Negotiation successful
0805 22:42:43.051281 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260805 22:42:43.052515 3552 raft_consensus.cc:1261] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Refusing update from remote peer 9818810f23fa4edc8a81dcb8e8df37ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:43.052574 3150 raft_consensus.cc:1261] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Refusing update from remote peer 9818810f23fa4edc8a81dcb8e8df37ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:43.052812 3716 consensus_queue.cc:1035] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:43.052939 3666 consensus_queue.cc:1035] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:43.053722 3685 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: TxnStatusTablet state changed. Reason: New leader 9818810f23fa4edc8a81dcb8e8df37ca. Latest consensus state: current_term: 1 leader_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.053828 3685 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:43.053849 3669 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: TxnStatusTablet state changed. Reason: New leader 9818810f23fa4edc8a81dcb8e8df37ca. Latest consensus state: current_term: 1 leader_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.053936 3669 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:43.054299 3716 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.054376 3716 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: This TxnStatusTablet replica's current role is: LEADER
I20260805 22:42:43.054670 3669 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.054749 3669 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P 482b2f2bcc874a249c7a645859e69dcd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:43.055635 3666 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.055750 3666 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P 9818810f23fa4edc8a81dcb8e8df37ca: This TxnStatusTablet replica's current role is: LEADER
I20260805 22:42:43.056154 3676 mvcc.cc:205] Tried to move back new op lower bound from 7315332149460926464 to 7315332149422063616. Current Snapshot: MvccSnapshot[applied={T|T < 7315332149460926464}]
I20260805 22:42:43.056250 3685 tablet_replica.cc:439] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } } }
I20260805 22:42:43.056315 3685 tablet_replica.cc:442] T 64daef816b254891ae909841960aa294 P afa13bd8003540aeabc0ed49805b1610: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 22:42:43.058368 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.055679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.23.197.67:44609)
0805 22:42:43.055725 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.055727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.055837 (+ 110us) server_negotiation.cc:407] Connection header received
0805 22:42:43.055876 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.055876 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.055901 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.055962 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:43.056092 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.056795 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.057567 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.057779 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.057854 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.058014 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.058115 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.058203 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.058251 (+ 48us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.058272 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:42:43.063118 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.060725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:40863 (local address 127.23.197.66:37765)
0805 22:42:43.060837 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.060839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.060954 (+ 115us) server_negotiation.cc:407] Connection header received
0805 22:42:43.061020 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.061021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.061031 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.061085 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:43.061222 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.061664 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.062425 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.062627 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.062700 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.062874 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.062905 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.062969 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.063031 (+ 62us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.063050 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260805 22:42:43.063149 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.060664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37765 (local address 127.23.197.67:40863)
0805 22:42:43.060888 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:43.060897 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:43.060970 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:43.061089 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:43.061090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:43.061091 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:43.061155 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.061157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.061749 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.061750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.062343 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.062344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.062715 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.062717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.062782 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.062796 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.062971 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.062974 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:42:43.063027 (+ 53us) client_negotiation.cc:240] Negotiation successful
0805 22:42:43.063066 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
I20260805 22:42:43.065270 3150 raft_consensus.cc:1261] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Refusing update from remote peer 482b2f2bcc874a249c7a645859e69dcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:43.065272 3352 raft_consensus.cc:1261] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Refusing update from remote peer 482b2f2bcc874a249c7a645859e69dcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:43.065508 3669 consensus_queue.cc:1035] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:43.065799 3701 consensus_queue.cc:1035] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:43.067385 3677 mvcc.cc:205] Tried to move back new op lower bound from 7315332149513891840 to 7315332149156286464. Current Snapshot: MvccSnapshot[applied={T|T < 7315332149513891840}]
I20260805 22:42:43.074182 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.071426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:38689 (local address 127.23.197.67:44609)
0805 22:42:43.071467 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.071469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.071619 (+ 150us) server_negotiation.cc:407] Connection header received
0805 22:42:43.071688 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.071688 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.071696 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.071741 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:43.071859 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.072304 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.073496 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.073765 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.073815 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.074003 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.074029 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.074066 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.074118 (+ 52us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.074133 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260805 22:42:43.074463 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.071393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:44609 (local address 127.23.197.66:38689)
0805 22:42:43.071565 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:43.071576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:43.071632 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:43.071744 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:43.071745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:43.071746 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:43.071805 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.071807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.072371 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.072372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.073416 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.073419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.073842 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.073843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.073910 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.073926 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.074067 (+ 141us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.074068 (+ 1us) client_negotiation.cc:769] Sending connection context
0805 22:42:43.074116 (+ 48us) client_negotiation.cc:240] Negotiation successful
0805 22:42:43.074156 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260805 22:42:43.083458 2936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:43.083580 2936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:43.084203 2936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728: Sending DeleteTablet for 3 replicas of tablet 24652459f3f34b979f60d6f1fc6853f7
I20260805 22:42:43.084473 3518 tablet_service.cc:1514] Processing DeleteTablet for tablet 24652459f3f34b979f60d6f1fc6853f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:43 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:60126
I20260805 22:42:43.084549 3121 tablet_service.cc:1514] Processing DeleteTablet for tablet 24652459f3f34b979f60d6f1fc6853f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:43 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:52194
I20260805 22:42:43.084667 3691 tablet_replica.cc:330] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: stopping tablet replica
I20260805 22:42:43.084817 3692 tablet_replica.cc:330] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: stopping tablet replica
I20260805 22:42:43.084801 3328 tablet_service.cc:1514] Processing DeleteTablet for tablet 24652459f3f34b979f60d6f1fc6853f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:43 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:58106
I20260805 22:42:43.084877 3692 raft_consensus.cc:2229] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:43.084913 3692 raft_consensus.cc:2258] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:43.084925 3694 tablet_replica.cc:330] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: stopping tablet replica
I20260805 22:42:43.084975 3694 raft_consensus.cc:2229] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:43.084735 3691 raft_consensus.cc:2229] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:43.085011 3694 raft_consensus.cc:2258] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:43.085084 3691 raft_consensus.cc:2258] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:43.085405 3692 ts_tablet_manager.cc:1905] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:43.085405 3694 ts_tablet_manager.cc:1905] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:43.087208 3691 ts_tablet_manager.cc:1905] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:43.088325 3694 ts_tablet_manager.cc:1918] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:43.088402 3694 log.cc:1196] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/24652459f3f34b979f60d6f1fc6853f7
I20260805 22:42:43.088626 3694 ts_tablet_manager.cc:1939] T 24652459f3f34b979f60d6f1fc6853f7 P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting consensus metadata
I20260805 22:42:43.088793 3691 ts_tablet_manager.cc:1918] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:43.088872 3691 log.cc:1196] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/24652459f3f34b979f60d6f1fc6853f7
I20260805 22:42:43.089017 2917 catalog_manager.cc:4841] TS 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765): tablet 24652459f3f34b979f60d6f1fc6853f7 (table test-table [id=d189f31ec57647e6a39d20d346f4eb88]) successfully deleted
I20260805 22:42:43.089111 3691 ts_tablet_manager.cc:1939] T 24652459f3f34b979f60d6f1fc6853f7 P 482b2f2bcc874a249c7a645859e69dcd: Deleting consensus metadata
I20260805 22:42:43.089294 3692 ts_tablet_manager.cc:1918] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:43.089361 3692 log.cc:1196] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/24652459f3f34b979f60d6f1fc6853f7
I20260805 22:42:43.089581 3692 ts_tablet_manager.cc:1939] T 24652459f3f34b979f60d6f1fc6853f7 P afa13bd8003540aeabc0ed49805b1610: Deleting consensus metadata
I20260805 22:42:43.089593 2920 catalog_manager.cc:4841] TS 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609): tablet 24652459f3f34b979f60d6f1fc6853f7 (table test-table [id=d189f31ec57647e6a39d20d346f4eb88]) successfully deleted
I20260805 22:42:43.089892 2922 catalog_manager.cc:4841] TS afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507): tablet 24652459f3f34b979f60d6f1fc6853f7 (table test-table [id=d189f31ec57647e6a39d20d346f4eb88]) successfully deleted
Aug 05 22:42:43 dist-test-slave-g170 krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 22:42:43 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:43 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
W20260805 22:42:43.124323 3007 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:43.115475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48174 (local address 127.23.197.126:35613)
0805 22:42:43.115526 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.115529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.116990 (+ 1461us) server_negotiation.cc:407] Connection header received
0805 22:42:43.117067 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.117068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.117084 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.117166 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:43.117932 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.118871 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.120176 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.120561 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.120648 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.124242 (+ 3594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48174: BlockingRecv error: recv got EOF from 127.0.0.1:48174 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 05 22:42:43 dist-test-slave-g170 krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM
I20260805 22:42:43.135251 3007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.125541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.23.197.126:35613)
0805 22:42:43.125596 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.125599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.125690 (+ 91us) server_negotiation.cc:407] Connection header received
0805 22:42:43.125751 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.125752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.125765 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.125842 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:43.126464 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.127378 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.128476 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.128866 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.128948 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.131863 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:43.131865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:43.131865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:43.131875 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:43.133812 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:43.134227 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:43.134228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:43.134229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:43.134317 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:43.134548 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:43.134549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:43.134550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:43.134772 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:43.134888 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.135042 (+ 154us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.135139 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260805 22:42:43.137077 2936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:43.137281 2936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:43.138728 3121 tablet_service.cc:1467] Processing CreateTablet for tablet 1717c24d71b44efdb285ab7867d7e07f (DEFAULT_TABLE table=test-table [id=28fd4364108f4d0a87bdd2e70548bbc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:43.138785 3328 tablet_service.cc:1467] Processing CreateTablet for tablet 1717c24d71b44efdb285ab7867d7e07f (DEFAULT_TABLE table=test-table [id=28fd4364108f4d0a87bdd2e70548bbc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:43.138883 3121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1717c24d71b44efdb285ab7867d7e07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:43.138931 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1717c24d71b44efdb285ab7867d7e07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:43.140242 3659 tablet_bootstrap.cc:492] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: Bootstrap starting.
I20260805 22:42:43.140381 3656 tablet_bootstrap.cc:492] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: Bootstrap starting.
I20260805 22:42:43.140569 3659 tablet_bootstrap.cc:654] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:43.140746 3656 tablet_bootstrap.cc:654] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:43.141491 3518 tablet_service.cc:1467] Processing CreateTablet for tablet 1717c24d71b44efdb285ab7867d7e07f (DEFAULT_TABLE table=test-table [id=28fd4364108f4d0a87bdd2e70548bbc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:43.141532 3659 tablet_bootstrap.cc:492] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: No bootstrap required, opened a new log
I20260805 22:42:43.141579 3659 ts_tablet_manager.cc:1397] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:43.141649 3518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1717c24d71b44efdb285ab7867d7e07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:43.141691 3659 raft_consensus.cc:348] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.141762 3659 raft_consensus.cc:374] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:43.141783 3659 raft_consensus.cc:729] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa13bd8003540aeabc0ed49805b1610, State: Initialized, Role: FOLLOWER
I20260805 22:42:43.141829 3659 consensus_queue.cc:260] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.141916 3659 ts_tablet_manager.cc:1428] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:43.143064 3657 tablet_bootstrap.cc:492] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: Bootstrap starting.
I20260805 22:42:43.143575 3657 tablet_bootstrap.cc:654] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:43.144402 3657 tablet_bootstrap.cc:492] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: No bootstrap required, opened a new log
I20260805 22:42:43.144459 3657 ts_tablet_manager.cc:1397] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:43.144687 3657 raft_consensus.cc:348] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.144805 3657 raft_consensus.cc:374] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:43.144831 3657 raft_consensus.cc:729] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b2f2bcc874a249c7a645859e69dcd, State: Initialized, Role: FOLLOWER
I20260805 22:42:43.144878 3657 consensus_queue.cc:260] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.144974 3657 ts_tablet_manager.cc:1428] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:42:43.145907 3656 tablet_bootstrap.cc:492] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: No bootstrap required, opened a new log
I20260805 22:42:43.145972 3656 ts_tablet_manager.cc:1397] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260805 22:42:43.146123 3656 raft_consensus.cc:348] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.146214 3656 raft_consensus.cc:374] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:43.146250 3656 raft_consensus.cc:729] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9818810f23fa4edc8a81dcb8e8df37ca, State: Initialized, Role: FOLLOWER
I20260805 22:42:43.146293 3656 consensus_queue.cc:260] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.146390 3656 ts_tablet_manager.cc:1428] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260805 22:42:43.390231 3725 raft_consensus.cc:482] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:43.390381 3725 raft_consensus.cc:504] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.390583 3725 leader_election.cc:288] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507), 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609)
I20260805 22:42:43.390857 3552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1717c24d71b44efdb285ab7867d7e07f" candidate_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b2f2bcc874a249c7a645859e69dcd" is_pre_election: true
I20260805 22:42:43.390857 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1717c24d71b44efdb285ab7867d7e07f" candidate_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa13bd8003540aeabc0ed49805b1610" is_pre_election: true
I20260805 22:42:43.390988 3552 raft_consensus.cc:2454] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9818810f23fa4edc8a81dcb8e8df37ca in term 0.
I20260805 22:42:43.390988 3150 raft_consensus.cc:2454] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9818810f23fa4edc8a81dcb8e8df37ca in term 0.
I20260805 22:42:43.391191 3252 leader_election.cc:302] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b2f2bcc874a249c7a645859e69dcd, 9818810f23fa4edc8a81dcb8e8df37ca; no voters:
I20260805 22:42:43.391330 3725 raft_consensus.cc:2781] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:43.391382 3725 raft_consensus.cc:482] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:43.391400 3725 raft_consensus.cc:3037] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:43.392220 3725 raft_consensus.cc:504] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.392376 3725 leader_election.cc:288] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [CANDIDATE]: Term 1 election: Requested vote from peers afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507), 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609)
I20260805 22:42:43.392552 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1717c24d71b44efdb285ab7867d7e07f" candidate_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa13bd8003540aeabc0ed49805b1610"
I20260805 22:42:43.392553 3552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1717c24d71b44efdb285ab7867d7e07f" candidate_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b2f2bcc874a249c7a645859e69dcd"
I20260805 22:42:43.392647 3150 raft_consensus.cc:3037] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:43.392647 3552 raft_consensus.cc:3037] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:43.393404 3150 raft_consensus.cc:2454] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9818810f23fa4edc8a81dcb8e8df37ca in term 1.
I20260805 22:42:43.393419 3552 raft_consensus.cc:2454] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9818810f23fa4edc8a81dcb8e8df37ca in term 1.
I20260805 22:42:43.393642 3252 leader_election.cc:302] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b2f2bcc874a249c7a645859e69dcd, 9818810f23fa4edc8a81dcb8e8df37ca; no voters:
I20260805 22:42:43.393734 3725 raft_consensus.cc:2781] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:43.393793 3725 raft_consensus.cc:686] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 LEADER]: Becoming Leader. State: Replica: 9818810f23fa4edc8a81dcb8e8df37ca, State: Running, Role: LEADER
I20260805 22:42:43.393864 3725 consensus_queue.cc:237] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.394511 2936 catalog_manager.cc:5495] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66). New cstate: current_term: 1 leader_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:43.430213 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.428100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.23.197.66:37765)
0805 22:42:43.428169 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.428171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.428179 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:43.428202 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.428203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.428213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.428272 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:43.428427 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.428907 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.429637 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.429825 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.429878 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.430016 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.430043 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.430082 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.430133 (+ 51us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.430152 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 22:42:43.431176 3552 raft_consensus.cc:1261] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Refusing update from remote peer 9818810f23fa4edc8a81dcb8e8df37ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:43.431195 3150 raft_consensus.cc:1261] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Refusing update from remote peer 9818810f23fa4edc8a81dcb8e8df37ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:43.431416 3666 consensus_queue.cc:1035] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:43.431560 3725 consensus_queue.cc:1035] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:43.435189 2936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:43.435279 2936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:43.435921 2936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728: Sending DeleteTablet for 3 replicas of tablet 1717c24d71b44efdb285ab7867d7e07f
I20260805 22:42:43.436220 3328 tablet_service.cc:1514] Processing DeleteTablet for tablet 1717c24d71b44efdb285ab7867d7e07f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:43 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:58106
I20260805 22:42:43.436354 3121 tablet_service.cc:1514] Processing DeleteTablet for tablet 1717c24d71b44efdb285ab7867d7e07f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:43 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:52194
I20260805 22:42:43.436453 3518 tablet_service.cc:1514] Processing DeleteTablet for tablet 1717c24d71b44efdb285ab7867d7e07f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:43 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:60126
I20260805 22:42:43.436550 3694 tablet_replica.cc:330] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: stopping tablet replica
I20260805 22:42:43.436619 3694 raft_consensus.cc:2229] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:43.436544 2936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:42:43.436687 2936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:42:43.436733 3694 raft_consensus.cc:2258] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:43.436834 3692 tablet_replica.cc:330] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: stopping tablet replica
I20260805 22:42:43.436895 3692 raft_consensus.cc:2229] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:43.436930 3692 raft_consensus.cc:2258] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:43.437001 3691 tablet_replica.cc:330] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: stopping tablet replica
I20260805 22:42:43.437067 3691 raft_consensus.cc:2229] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:43.437119 3691 raft_consensus.cc:2258] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:43.437542 3694 ts_tablet_manager.cc:1905] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:43.437610 3692 ts_tablet_manager.cc:1905] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:43.437988 3328 tablet_service.cc:1467] Processing CreateTablet for tablet 2910252872664af89e87ae9a1d8ac05f (DEFAULT_TABLE table=test-table [id=cc853b2a945f42bc95e0b75f2330737a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:43.438092 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2910252872664af89e87ae9a1d8ac05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:43.439291 3691 ts_tablet_manager.cc:1905] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:43.439677 3121 tablet_service.cc:1467] Processing CreateTablet for tablet 2910252872664af89e87ae9a1d8ac05f (DEFAULT_TABLE table=test-table [id=cc853b2a945f42bc95e0b75f2330737a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:43.439852 3656 tablet_bootstrap.cc:492] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: Bootstrap starting.
I20260805 22:42:43.440187 3656 tablet_bootstrap.cc:654] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:43.440557 3692 ts_tablet_manager.cc:1918] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:43.440608 3692 log.cc:1196] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/1717c24d71b44efdb285ab7867d7e07f
I20260805 22:42:43.440846 3121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2910252872664af89e87ae9a1d8ac05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:43.440904 3694 ts_tablet_manager.cc:1918] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:43.440960 3694 log.cc:1196] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/1717c24d71b44efdb285ab7867d7e07f
I20260805 22:42:43.441008 3691 ts_tablet_manager.cc:1918] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:42:43.441058 3691 log.cc:1196] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/1717c24d71b44efdb285ab7867d7e07f
I20260805 22:42:43.441196 3694 ts_tablet_manager.cc:1939] T 1717c24d71b44efdb285ab7867d7e07f P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting consensus metadata
I20260805 22:42:43.441279 3691 ts_tablet_manager.cc:1939] T 1717c24d71b44efdb285ab7867d7e07f P 482b2f2bcc874a249c7a645859e69dcd: Deleting consensus metadata
I20260805 22:42:43.441663 3518 tablet_service.cc:1467] Processing CreateTablet for tablet 2910252872664af89e87ae9a1d8ac05f (DEFAULT_TABLE table=test-table [id=cc853b2a945f42bc95e0b75f2330737a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:42:43.441766 3518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2910252872664af89e87ae9a1d8ac05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:43.442252 3659 tablet_bootstrap.cc:492] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610: Bootstrap starting.
I20260805 22:42:43.442494 2917 catalog_manager.cc:4841] TS 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765): tablet 1717c24d71b44efdb285ab7867d7e07f (table test-table [id=28fd4364108f4d0a87bdd2e70548bbc6]) successfully deleted
I20260805 22:42:43.442565 3659 tablet_bootstrap.cc:654] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:43.443009 3657 tablet_bootstrap.cc:492] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: Bootstrap starting.
I20260805 22:42:43.443011 3656 tablet_bootstrap.cc:492] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: No bootstrap required, opened a new log
I20260805 22:42:43.443069 3656 ts_tablet_manager.cc:1397] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260805 22:42:43.443220 3656 raft_consensus.cc:348] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.443292 3656 raft_consensus.cc:374] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:43.443307 3656 raft_consensus.cc:729] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9818810f23fa4edc8a81dcb8e8df37ca, State: Initialized, Role: FOLLOWER
I20260805 22:42:43.443349 3657 tablet_bootstrap.cc:654] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:43.443379 3656 consensus_queue.cc:260] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.443475 3656 ts_tablet_manager.cc:1428] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:43.443776 3659 tablet_bootstrap.cc:492] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610: No bootstrap required, opened a new log
I20260805 22:42:43.443825 3659 ts_tablet_manager.cc:1397] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:42:43.443926 3659 raft_consensus.cc:348] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.443988 3659 raft_consensus.cc:374] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:43.444006 3659 raft_consensus.cc:729] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa13bd8003540aeabc0ed49805b1610, State: Initialized, Role: FOLLOWER
I20260805 22:42:43.444042 3659 consensus_queue.cc:260] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.444119 3659 ts_tablet_manager.cc:1428] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:43.444312 3692 ts_tablet_manager.cc:1939] T 1717c24d71b44efdb285ab7867d7e07f P afa13bd8003540aeabc0ed49805b1610: Deleting consensus metadata
I20260805 22:42:43.444585 3657 tablet_bootstrap.cc:492] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: No bootstrap required, opened a new log
I20260805 22:42:43.444633 3657 ts_tablet_manager.cc:1397] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:42:43.444693 2922 catalog_manager.cc:4841] TS afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507): tablet 1717c24d71b44efdb285ab7867d7e07f (table test-table [id=28fd4364108f4d0a87bdd2e70548bbc6]) successfully deleted
I20260805 22:42:43.444780 3657 raft_consensus.cc:348] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.444849 3657 raft_consensus.cc:374] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:43.444869 3657 raft_consensus.cc:729] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b2f2bcc874a249c7a645859e69dcd, State: Initialized, Role: FOLLOWER
I20260805 22:42:43.444916 3657 consensus_queue.cc:260] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.445004 3657 ts_tablet_manager.cc:1428] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:43.445286 2920 catalog_manager.cc:4841] TS 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609): tablet 1717c24d71b44efdb285ab7867d7e07f (table test-table [id=28fd4364108f4d0a87bdd2e70548bbc6]) successfully deleted
I20260805 22:42:43.584726 3685 raft_consensus.cc:482] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:42:43.584906 3685 raft_consensus.cc:504] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.585084 3685 leader_election.cc:288] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765), 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609)
I20260805 22:42:43.585355 3552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2910252872664af89e87ae9a1d8ac05f" candidate_uuid: "afa13bd8003540aeabc0ed49805b1610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b2f2bcc874a249c7a645859e69dcd" is_pre_election: true
I20260805 22:42:43.585350 3352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2910252872664af89e87ae9a1d8ac05f" candidate_uuid: "afa13bd8003540aeabc0ed49805b1610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" is_pre_election: true
I20260805 22:42:43.585450 3552 raft_consensus.cc:2454] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa13bd8003540aeabc0ed49805b1610 in term 0.
I20260805 22:42:43.585464 3352 raft_consensus.cc:2454] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa13bd8003540aeabc0ed49805b1610 in term 0.
I20260805 22:42:43.585620 3062 leader_election.cc:302] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b2f2bcc874a249c7a645859e69dcd, afa13bd8003540aeabc0ed49805b1610; no voters:
I20260805 22:42:43.585752 3685 raft_consensus.cc:2781] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:42:43.585803 3685 raft_consensus.cc:482] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:42:43.585816 3685 raft_consensus.cc:3037] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:43.586565 3685 raft_consensus.cc:504] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.586710 3685 leader_election.cc:288] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [CANDIDATE]: Term 1 election: Requested vote from peers 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765), 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609)
I20260805 22:42:43.586898 3352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2910252872664af89e87ae9a1d8ac05f" candidate_uuid: "afa13bd8003540aeabc0ed49805b1610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9818810f23fa4edc8a81dcb8e8df37ca"
I20260805 22:42:43.586973 3352 raft_consensus.cc:3037] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:43.586928 3552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2910252872664af89e87ae9a1d8ac05f" candidate_uuid: "afa13bd8003540aeabc0ed49805b1610" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b2f2bcc874a249c7a645859e69dcd"
I20260805 22:42:43.587013 3552 raft_consensus.cc:3037] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:43.587623 3352 raft_consensus.cc:2454] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa13bd8003540aeabc0ed49805b1610 in term 1.
I20260805 22:42:43.587689 3552 raft_consensus.cc:2454] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa13bd8003540aeabc0ed49805b1610 in term 1.
I20260805 22:42:43.587821 3061 leader_election.cc:302] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9818810f23fa4edc8a81dcb8e8df37ca, afa13bd8003540aeabc0ed49805b1610; no voters:
I20260805 22:42:43.587952 3685 raft_consensus.cc:2781] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:43.588085 3685 raft_consensus.cc:686] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 1 LEADER]: Becoming Leader. State: Replica: afa13bd8003540aeabc0ed49805b1610, State: Running, Role: LEADER
I20260805 22:42:43.588148 3685 consensus_queue.cc:237] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } }
I20260805 22:42:43.588716 2934 catalog_manager.cc:5495] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 reported cstate change: term changed from 0 to 1, leader changed from <none> to afa13bd8003540aeabc0ed49805b1610 (127.23.197.65). New cstate: current_term: 1 leader_uuid: "afa13bd8003540aeabc0ed49805b1610" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa13bd8003540aeabc0ed49805b1610" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 39507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 } health_report { overall_health: UNKNOWN } } }
I20260805 22:42:43.595445 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.593172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52198 (local address 127.23.197.65:39507)
0805 22:42:43.593323 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.593326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.593333 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:42:43.593354 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.593355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.593365 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.593440 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:43.593549 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.594044 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.594770 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.594994 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.595072 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.595195 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.595229 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.595276 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.595330 (+ 54us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.595353 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260805 22:42:43.600579 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.598242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:47627 (local address 127.23.197.66:37765)
0805 22:42:43.598318 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.598329 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.598485 (+ 156us) server_negotiation.cc:407] Connection header received
0805 22:42:43.598548 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.598549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.598557 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.598608 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:43.598719 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.599163 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.599984 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.600175 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.600256 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.600393 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.600419 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.600458 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.600509 (+ 51us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.600524 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 22:42:43.600597 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.598174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37765 (local address 127.23.197.65:47627)
0805 22:42:43.598413 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:43.598427 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:43.598481 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:43.598624 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:43.598624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:43.598626 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:43.598673 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.598674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.599242 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.599243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.599895 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.599897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.600260 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.600261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.600305 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.600321 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.600468 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.600470 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:42:43.600501 (+ 31us) client_negotiation.cc:240] Negotiation successful
0805 22:42:43.600540 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":110,"threads_started":1}
I20260805 22:42:43.602828 3552 raft_consensus.cc:1261] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Refusing update from remote peer afa13bd8003540aeabc0ed49805b1610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:43.602854 3352 raft_consensus.cc:1261] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Refusing update from remote peer afa13bd8003540aeabc0ed49805b1610: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:42:43.603044 3769 consensus_queue.cc:1035] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482b2f2bcc874a249c7a645859e69dcd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:43.603145 3685 consensus_queue.cc:1035] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9818810f23fa4edc8a81dcb8e8df37ca" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:42:43.609980 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.607769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:38177 (local address 127.23.197.65:39507)
0805 22:42:43.607809 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:43.607811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:43.607819 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:43.607883 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:43.607884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:43.607893 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:43.607946 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:42:43.608052 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.608517 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.609302 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.609508 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.609590 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.609734 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.609766 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.609829 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:43.609890 (+ 61us) server_negotiation.cc:299] Negotiation successful
0805 22:42:43.609912 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:42:43.610015 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.607655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:39507 (local address 127.23.197.66:38177)
0805 22:42:43.607753 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:43.607766 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:43.607821 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:43.607950 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:43.607951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:43.607952 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 22:42:43.607996 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.607998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.608595 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.608596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.609213 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:43.609215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:43.609592 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:43.609593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:43.609649 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:43.609653 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 22:42:43.609830 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 22:42:43.609832 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:42:43.609885 (+ 53us) client_negotiation.cc:240] Negotiation successful
0805 22:42:43.609935 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":60,"threads_started":1}
I20260805 22:42:43.615861 2934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:43.615957 2934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:42:43.616566 2934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26d90433919d4510afad78ad906cf728: Sending DeleteTablet for 3 replicas of tablet 2910252872664af89e87ae9a1d8ac05f
I20260805 22:42:43.616832 3328 tablet_service.cc:1514] Processing DeleteTablet for tablet 2910252872664af89e87ae9a1d8ac05f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:43 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:58106
I20260805 22:42:43.616833 3121 tablet_service.cc:1514] Processing DeleteTablet for tablet 2910252872664af89e87ae9a1d8ac05f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:43 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:52194
I20260805 22:42:43.616876 3518 tablet_service.cc:1514] Processing DeleteTablet for tablet 2910252872664af89e87ae9a1d8ac05f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:42:43 UTC) from {username='oryx', principal='oryx/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:60126
I20260805 22:42:43.616927 3692 tablet_replica.cc:330] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610: stopping tablet replica
I20260805 22:42:43.616981 3692 raft_consensus.cc:2229] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:42:43.616984 3691 tablet_replica.cc:330] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: stopping tablet replica
I20260805 22:42:43.616990 3694 tablet_replica.cc:330] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: stopping tablet replica
I20260805 22:42:43.617038 3692 raft_consensus.cc:2258] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:43.617043 3691 raft_consensus.cc:2229] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:43.617049 3694 raft_consensus.cc:2229] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:42:43.617082 3694 raft_consensus.cc:2258] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:43.617090 3691 raft_consensus.cc:2258] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:42:43.617228 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 3036
I20260805 22:42:43.617532 3691 ts_tablet_manager.cc:1905] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:43.617591 3692 ts_tablet_manager.cc:1905] T 2910252872664af89e87ae9a1d8ac05f P afa13bd8003540aeabc0ed49805b1610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:43.617660 3694 ts_tablet_manager.cc:1905] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:42:43.619220 3691 ts_tablet_manager.cc:1918] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:43.619277 3691 log.cc:1196] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/2910252872664af89e87ae9a1d8ac05f
I20260805 22:42:43.619460 3691 ts_tablet_manager.cc:1939] T 2910252872664af89e87ae9a1d8ac05f P 482b2f2bcc874a249c7a645859e69dcd: Deleting consensus metadata
I20260805 22:42:43.619696 3694 ts_tablet_manager.cc:1918] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 22:42:43.619769 3694 log.cc:1196] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/2910252872664af89e87ae9a1d8ac05f
I20260805 22:42:43.619985 3694 ts_tablet_manager.cc:1939] T 2910252872664af89e87ae9a1d8ac05f P 9818810f23fa4edc8a81dcb8e8df37ca: Deleting consensus metadata
I20260805 22:42:43.620012 2920 catalog_manager.cc:4841] TS 482b2f2bcc874a249c7a645859e69dcd (127.23.197.67:44609): tablet 2910252872664af89e87ae9a1d8ac05f (table test-table [id=cc853b2a945f42bc95e0b75f2330737a]) successfully deleted
I20260805 22:42:43.620297 2917 catalog_manager.cc:4841] TS 9818810f23fa4edc8a81dcb8e8df37ca (127.23.197.66:37765): tablet 2910252872664af89e87ae9a1d8ac05f (table test-table [id=cc853b2a945f42bc95e0b75f2330737a]) successfully deleted
W20260805 22:42:43.632273 2922 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:39507 (error 108)
I20260805 22:42:43.632740 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 3226
I20260805 22:42:43.632980 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:43.632722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:39507 (local address 127.0.0.1:52212)
0805 22:42:43.632856 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:43.632923 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:39507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
W20260805 22:42:43.637132 2922 catalog_manager.cc:4584] TS afa13bd8003540aeabc0ed49805b1610 (127.23.197.65:39507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2910252872664af89e87ae9a1d8ac05f: Network error: Client connection negotiation failed: client connection to 127.23.197.65:39507: connect: Connection refused (error 111)
I20260805 22:42:43.648980 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 3428
I20260805 22:42:43.667128 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 2893
2026-08-05T22:42:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3227 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:43 dist-test-slave-g170 krb5kdc[3794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:43 dist-test-slave-g170 krb5kdc[3794](info): set up 2 sockets
Aug 05 22:42:43 dist-test-slave-g170 krb5kdc[3794](info): commencing operation
krb5kdc: starting...
W20260805 22:42:44.992311 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:45 dist-test-slave-g170 krb5kdc[3794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:45.128191 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38403
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:38403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:45.213871 3809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:45.214133 3809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:45.214174 3809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:45.215322 3809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:45.215392 3809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:45.215415 3809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:45.215436 3809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:45.215466 3809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:45.217022 3809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:38403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:45.217314 3809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:45.217552 3809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 22:42:45.222889 3809 server_base.cc:1034] running on GCE node
W20260805 22:42:45.222856 3819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:45.222891 3823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:45.222856 3817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:45.223332 3809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:45.223687 3809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:45.224897 3809 hybrid_clock.cc:648] HybridClock initialized: now 1785969765224857 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-05T22:42:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1562 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:42:45 dist-test-slave-g170 krb5kdc[3829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:42:45 dist-test-slave-g170 krb5kdc[3829](info): set up 2 sockets
Aug 05 22:42:45 dist-test-slave-g170 krb5kdc[3829](info): commencing operation
krb5kdc: starting...
W20260805 22:42:46.501960 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:42:46 dist-test-slave-g170 krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:42:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:42:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:42:46.644476 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:39127
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:39127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:46.738389 3850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:46.738685 3850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:46.738729 3850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:46.740062 3850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:46.740144 3850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:46.740168 3850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:46.740191 3850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:46.740227 3850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:46.742082 3850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:39127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:39127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:46.742436 3850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:46.742715 3850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:46.748586 3858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:46.748591 3864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:46.748637 3860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:46.748888 3850 server_base.cc:1034] running on GCE node
I20260805 22:42:46.749190 3850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:46.749526 3850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:46.750720 3850 hybrid_clock.cc:648] HybridClock initialized: now 1785969766750695 us; error 49 us; skew 500 ppm
Aug 05 22:42:46 dist-test-slave-g170 krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969766, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:46.753595 3850 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:46.754137 3850 webserver.cc:460] Webserver started at http://127.23.197.126:39633/ using document root <none> and password file <none>
I20260805 22:42:46.754405 3850 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:46.754457 3850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:46.754652 3850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:46.755875 3850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "d2b07efbc39348fc98cb37d6b5554e84"
format_stamp: "Formatted at 2026-08-05 22:42:46 on dist-test-slave-g170"
server_key: "b16338a401e10656397db6b57340991f"
server_key_iv: "b1d0522d2bdb9fe3e5e08c6be8ae253e"
server_key_version: "encryptionkey@0"
I20260805 22:42:46.756296 3850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "d2b07efbc39348fc98cb37d6b5554e84"
format_stamp: "Formatted at 2026-08-05 22:42:46 on dist-test-slave-g170"
server_key: "b16338a401e10656397db6b57340991f"
server_key_iv: "b1d0522d2bdb9fe3e5e08c6be8ae253e"
server_key_version: "encryptionkey@0"
I20260805 22:42:46.757989 3850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260805 22:42:46.758929 3875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:46.759143 3850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:46.759268 3850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "d2b07efbc39348fc98cb37d6b5554e84"
format_stamp: "Formatted at 2026-08-05 22:42:46 on dist-test-slave-g170"
server_key: "b16338a401e10656397db6b57340991f"
server_key_iv: "b1d0522d2bdb9fe3e5e08c6be8ae253e"
server_key_version: "encryptionkey@0"
I20260805 22:42:46.759358 3850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:46.772056 3850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:46.774916 3850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:46.775106 3850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:46.779568 3850 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:39127
I20260805 22:42:46.779609 3946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:39127 every 8 connection(s)
I20260805 22:42:46.779990 3850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:46.781028 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 3850
I20260805 22:42:46.781044 3947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:42:46.781183 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:42:46.781390 24341 external_mini_cluster.cc:1428] Setting key 9b49128e2bcb2c7c13579c9f596ab335
I20260805 22:42:46.784569 3947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: Bootstrap starting.
I20260805 22:42:46.785411 3947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: Neither blocks nor log segments found. Creating new log.
I20260805 22:42:46.785727 3947 log.cc:824] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:46.786676 3947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: No bootstrap required, opened a new log
Aug 05 22:42:46 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:46.789464 3947 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } }
I20260805 22:42:46.789628 3947 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:42:46.789675 3947 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b07efbc39348fc98cb37d6b5554e84, State: Initialized, Role: FOLLOWER
I20260805 22:42:46.789813 3947 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } }
I20260805 22:42:46.789921 3947 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:46.789976 3947 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:46.790026 3947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:42:46.790628 3947 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } }
I20260805 22:42:46.790758 3947 leader_election.cc:302] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2b07efbc39348fc98cb37d6b5554e84; no voters:
I20260805 22:42:46.790951 3947 leader_election.cc:288] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:42:46.791008 3955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:42:46.791168 3955 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 1 LEADER]: Becoming Leader. State: Replica: d2b07efbc39348fc98cb37d6b5554e84, State: Running, Role: LEADER
I20260805 22:42:46.791280 3947 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:46.791291 3955 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } }
I20260805 22:42:46.791644 3955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2b07efbc39348fc98cb37d6b5554e84. Latest consensus state: current_term: 1 leader_uuid: "d2b07efbc39348fc98cb37d6b5554e84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } } }
I20260805 22:42:46.791759 3955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:46.791653 3960 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2b07efbc39348fc98cb37d6b5554e84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } } }
I20260805 22:42:46.791919 3960 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:46.792040 3968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:46.792610 3968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:46.794406 3968 catalog_manager.cc:1347] Generated new cluster ID: eb51fdeea5c341538d921aefbbb8046c
I20260805 22:42:46.794412 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:46.782079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.23.197.126:39127)
0805 22:42:46.782272 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:46.782280 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:46.782306 (+ 26us) server_negotiation.cc:407] Connection header received
0805 22:42:46.782755 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:46.782762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:46.782814 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:46.782946 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:46.783935 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:46.785011 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:46.786131 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:46.786855 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:46.786953 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:46.789832 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:46.789837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:46.789837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:46.789849 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:46.792090 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:46.792538 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:46.792542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:46.792542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:46.792712 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:46.793058 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:46.793059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:46.793060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:46.793378 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:46.793964 (+ 586us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:46.794044 (+ 80us) server_negotiation.cc:299] Negotiation successful
0805 22:42:46.794186 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260805 22:42:46.794468 3968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:46.811285 3968 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:42:46.812610 3968 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:46.821731 3968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: Generated new TSK 0
I20260805 22:42:46.821991 3968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:42:46.856175 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:39127
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:46.946913 3989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:46.947129 3989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:46.947155 3989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:46.948370 3989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:46.948416 3989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:46.948443 3989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:46.950013 3989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:46.950285 3989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:46.950551 3989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:46.955909 3996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:46.956115 3989 server_base.cc:1034] running on GCE node
W20260805 22:42:46.955950 4002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:46.955905 4000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:46.956423 3989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:46.956625 3989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:46.957791 3989 hybrid_clock.cc:648] HybridClock initialized: now 1785969766957797 us; error 49 us; skew 500 ppm
Aug 05 22:42:46 dist-test-slave-g170 krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969766, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:46.960440 3989 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:42:46.960958 3989 webserver.cc:460] Webserver started at http://127.23.197.65:43713/ using document root <none> and password file <none>
I20260805 22:42:46.961166 3989 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:46.961208 3989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:46.961301 3989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:46.962324 3989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "e05a8a9482bf4f4c93325450417751bc"
format_stamp: "Formatted at 2026-08-05 22:42:46 on dist-test-slave-g170"
server_key: "bf7f5c6ec4abcea1ea2629752bc16195"
server_key_iv: "c2eda84155464370fb1f88997bc91b1b"
server_key_version: "encryptionkey@0"
I20260805 22:42:46.962767 3989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "e05a8a9482bf4f4c93325450417751bc"
format_stamp: "Formatted at 2026-08-05 22:42:46 on dist-test-slave-g170"
server_key: "bf7f5c6ec4abcea1ea2629752bc16195"
server_key_iv: "c2eda84155464370fb1f88997bc91b1b"
server_key_version: "encryptionkey@0"
I20260805 22:42:46.964203 3989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:46.965124 4011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:46.965341 3989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:46.965458 3989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "e05a8a9482bf4f4c93325450417751bc"
format_stamp: "Formatted at 2026-08-05 22:42:46 on dist-test-slave-g170"
server_key: "bf7f5c6ec4abcea1ea2629752bc16195"
server_key_iv: "c2eda84155464370fb1f88997bc91b1b"
server_key_version: "encryptionkey@0"
I20260805 22:42:46.965540 3989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:46.987972 3989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:46.990715 3989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:46.990914 3989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:46.991214 3989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:46.991581 3989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:46.991639 3989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:46.991688 3989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:46.991714 3989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:46.998343 3989 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:35245
I20260805 22:42:46.998396 4168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:35245 every 8 connection(s)
I20260805 22:42:46.998778 3989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:42:47.003492 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 3989
I20260805 22:42:47.003620 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:42:47.003861 24341 external_mini_cluster.cc:1428] Setting key 95557644ee81e48bc00c035f01eb4bbf
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969766, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:47.009869 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:46.999709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:40331 (local address 127.23.197.126:39127)
0805 22:42:46.999769 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:46.999772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:46.999975 (+ 203us) server_negotiation.cc:407] Connection header received
0805 22:42:47.000457 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:47.000459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:47.000472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:47.000552 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:47.001583 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.002135 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.003365 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.003715 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.003806 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.006487 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:47.006489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:47.006490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:47.006499 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:47.008183 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:47.008672 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:47.008673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:47.008674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:47.008745 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:47.009079 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:47.009080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:47.009080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:47.009228 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:47.009312 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:47.009646 (+ 334us) server_negotiation.cc:299] Negotiation successful
0805 22:42:47.009759 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260805 22:42:47.010103 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:46.999643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:40331)
0805 22:42:46.999810 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:46.999854 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:47.000329 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:47.000604 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:47.000607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:47.001263 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:47.001464 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.001468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.002256 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.002257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.003246 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.003250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.003817 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.003818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.003985 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.004679 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:47.004696 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:47.006366 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:47.008311 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:47.008320 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:47.008322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:47.008579 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:47.008853 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:47.008854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:47.008854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:47.008998 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:47.009317 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:47.009325 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:47.009485 (+ 160us) client_negotiation.cc:769] Sending connection context
0805 22:42:47.009605 (+ 120us) client_negotiation.cc:240] Negotiation successful
0805 22:42:47.009818 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260805 22:42:47.010732 4170 heartbeater.cc:347] Connected to a master server at 127.23.197.126:39127
I20260805 22:42:47.010828 4170 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:47.011059 4170 heartbeater.cc:511] Master 127.23.197.126:39127 requested a full tablet report, sending...
I20260805 22:42:47.011745 3894 ts_manager.cc:194] Registered new tserver with Master: e05a8a9482bf4f4c93325450417751bc (127.23.197.65:35245)
I20260805 22:42:47.012743 3894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:40331
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:42:47.036410 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:39127
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:47.121969 4180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:47.122242 4180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:47.122282 4180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:47.123517 4180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:47.123590 4180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:47.123632 4180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:47.125223 4180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:47.125520 4180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:47.125901 4180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:47.131582 4186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:47.131568 4189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:47.131567 4187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:47.132073 4180 server_base.cc:1034] running on GCE node
I20260805 22:42:47.132287 4180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:47.132520 4180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:47.133708 4180 hybrid_clock.cc:648] HybridClock initialized: now 1785969767133670 us; error 48 us; skew 500 ppm
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:47.136559 4180 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:42:47.137073 4180 webserver.cc:460] Webserver started at http://127.23.197.66:40389/ using document root <none> and password file <none>
I20260805 22:42:47.137301 4180 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:47.137346 4180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:47.137441 4180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:47.138505 4180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "35df061a072348c2b72838407578dd41"
format_stamp: "Formatted at 2026-08-05 22:42:47 on dist-test-slave-g170"
server_key: "3a21aaaa3f23b582e0534b477af3e695"
server_key_iv: "b7818a0add4878516e7f567a9aa7d352"
server_key_version: "encryptionkey@0"
I20260805 22:42:47.138811 4180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "35df061a072348c2b72838407578dd41"
format_stamp: "Formatted at 2026-08-05 22:42:47 on dist-test-slave-g170"
server_key: "3a21aaaa3f23b582e0534b477af3e695"
server_key_iv: "b7818a0add4878516e7f567a9aa7d352"
server_key_version: "encryptionkey@0"
I20260805 22:42:47.140177 4180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:42:47.141120 4196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.141366 4180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:47.141494 4180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "35df061a072348c2b72838407578dd41"
format_stamp: "Formatted at 2026-08-05 22:42:47 on dist-test-slave-g170"
server_key: "3a21aaaa3f23b582e0534b477af3e695"
server_key_iv: "b7818a0add4878516e7f567a9aa7d352"
server_key_version: "encryptionkey@0"
I20260805 22:42:47.141585 4180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:47.152740 4180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:47.155534 4180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:47.155736 4180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:47.156013 4180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:47.156383 4180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:47.156445 4180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.156497 4180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:47.156524 4180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.163607 4180 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:41729
I20260805 22:42:47.163735 4363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:41729 every 8 connection(s)
I20260805 22:42:47.164062 4180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:47.172806 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 4180
I20260805 22:42:47.172948 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:42:47.173172 24341 external_mini_cluster.cc:1428] Setting key 100b808015099fa8ca79616d50d9ccbf
I20260805 22:42:47.174860 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.164994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:36671)
0805 22:42:47.165213 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:47.165253 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:47.165770 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:47.165990 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:47.165993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:47.166595 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:47.166812 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.166816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.167513 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.167514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.168350 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.168353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.168912 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.168913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.169028 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.169679 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:47.169688 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:47.171121 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:47.173018 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:47.173026 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:47.173029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:47.173339 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:47.173701 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:47.173702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:47.173703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:47.173833 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:47.174334 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:47.174340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:47.174464 (+ 124us) client_negotiation.cc:769] Sending connection context
0805 22:42:47.174554 (+ 90us) client_negotiation.cc:240] Negotiation successful
0805 22:42:47.174677 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":126,"threads_started":1}
I20260805 22:42:47.174873 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.165015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:36671 (local address 127.23.197.126:39127)
0805 22:42:47.165116 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:47.165119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:47.165360 (+ 241us) server_negotiation.cc:407] Connection header received
0805 22:42:47.165859 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:47.165860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:47.165878 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:47.165933 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:47.166907 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.167404 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.168458 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.168815 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.168900 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.171215 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:47.171217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:47.171217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:47.171224 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:47.172930 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:47.173534 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:47.173536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:47.173536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:47.173612 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:47.174070 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:47.174072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:47.174072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:47.174251 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:47.174338 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:47.174604 (+ 266us) server_negotiation.cc:299] Negotiation successful
0805 22:42:47.174706 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 22:42:47.175345 4364 heartbeater.cc:347] Connected to a master server at 127.23.197.126:39127
I20260805 22:42:47.175464 4364 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:47.175676 4364 heartbeater.cc:511] Master 127.23.197.126:39127 requested a full tablet report, sending...
I20260805 22:42:47.176186 3894 ts_manager.cc:194] Registered new tserver with Master: 35df061a072348c2b72838407578dd41 (127.23.197.66:41729)
I20260805 22:42:47.176743 3894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:36671
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:42:47.204273 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:39127
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:47.296715 4369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:47.296993 4369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:47.297017 4369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:47.298161 4369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:47.298197 4369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:47.298223 4369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:47.299620 4369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:47.299810 4369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:47.300030 4369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:47.305095 4378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:47.305130 4376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:47.305141 4374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:47.305576 4369 server_base.cc:1034] running on GCE node
I20260805 22:42:47.305836 4369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:47.306032 4369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:47.307185 4369 hybrid_clock.cc:648] HybridClock initialized: now 1785969767307170 us; error 34 us; skew 500 ppm
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:47.309949 4369 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:42:47.310436 4369 webserver.cc:460] Webserver started at http://127.23.197.67:36583/ using document root <none> and password file <none>
I20260805 22:42:47.310674 4369 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:47.310720 4369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:47.310869 4369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:42:47.312000 4369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "adba40d0cd04430a9622fac9876018e4"
format_stamp: "Formatted at 2026-08-05 22:42:47 on dist-test-slave-g170"
server_key: "53e3d22a7ec6c6a5561e4becf77bdd11"
server_key_iv: "99dac8b1e1b7f11a86cbe55311f5b1a5"
server_key_version: "encryptionkey@0"
I20260805 22:42:47.312376 4369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "adba40d0cd04430a9622fac9876018e4"
format_stamp: "Formatted at 2026-08-05 22:42:47 on dist-test-slave-g170"
server_key: "53e3d22a7ec6c6a5561e4becf77bdd11"
server_key_iv: "99dac8b1e1b7f11a86cbe55311f5b1a5"
server_key_version: "encryptionkey@0"
I20260805 22:42:47.313901 4369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:42:47.314695 4387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.314924 4369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:47.315035 4369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "adba40d0cd04430a9622fac9876018e4"
format_stamp: "Formatted at 2026-08-05 22:42:47 on dist-test-slave-g170"
server_key: "53e3d22a7ec6c6a5561e4becf77bdd11"
server_key_iv: "99dac8b1e1b7f11a86cbe55311f5b1a5"
server_key_version: "encryptionkey@0"
I20260805 22:42:47.315101 4369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:47.321841 4369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:47.323863 4369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:47.324024 4369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:47.324257 4369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:47.324577 4369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:47.324644 4369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.324687 4369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:47.324713 4369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.331346 4369 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:32849
I20260805 22:42:47.331478 4543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:32849 every 8 connection(s)
I20260805 22:42:47.331771 4369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:42:47.340914 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 4369
I20260805 22:42:47.341048 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:42:47.341295 24341 external_mini_cluster.cc:1428] Setting key 79c9f80054ecec8f7c3461c6dd51f73b
I20260805 22:42:47.342322 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.332541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:40325 (local address 127.23.197.126:39127)
0805 22:42:47.332588 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:47.332590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:47.332823 (+ 233us) server_negotiation.cc:407] Connection header received
0805 22:42:47.333375 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:47.333376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:47.333390 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:47.333442 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:47.334436 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.334957 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.335996 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.336203 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.336277 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.338893 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:47.338894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:47.338894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:47.338902 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:47.340466 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:47.341182 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:47.341183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:47.341183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:47.341243 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:47.341590 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:47.341592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:47.341592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:47.341739 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:47.341829 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:47.342128 (+ 299us) server_negotiation.cc:299] Negotiation successful
0805 22:42:47.342221 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:42:47.342525 4546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.332507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:40325)
0805 22:42:47.332671 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:47.332715 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:47.333255 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:47.333511 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:47.333515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:47.334133 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:47.334330 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.334334 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.335068 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.335069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.335899 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.335901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.336301 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.336302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.336434 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.337111 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:47.337121 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:47.338773 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:42:47.340597 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:47.340608 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:47.340611 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:47.341085 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:47.341325 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:42:47.341326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:42:47.341326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:42:47.341495 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:42:47.341818 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:42:47.341826 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:42:47.341973 (+ 147us) client_negotiation.cc:769] Sending connection context
0805 22:42:47.342087 (+ 114us) client_negotiation.cc:240] Negotiation successful
0805 22:42:47.342285 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260805 22:42:47.343008 4544 heartbeater.cc:347] Connected to a master server at 127.23.197.126:39127
I20260805 22:42:47.343149 4544 heartbeater.cc:464] Registering TS with master...
I20260805 22:42:47.343361 4544 heartbeater.cc:511] Master 127.23.197.126:39127 requested a full tablet report, sending...
I20260805 22:42:47.343780 3894 ts_manager.cc:194] Registered new tserver with Master: adba40d0cd04430a9622fac9876018e4 (127.23.197.67:32849)
I20260805 22:42:47.344147 3894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:40325
I20260805 22:42:47.353391 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab.
Entry for principal oryx/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab.
WARNING: no policy specified for oryx/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab.
Entry for principal oryx/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab.
WARNING: no policy specified for oryx/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab.
Entry for principal oryx/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab.
I20260805 22:42:47.441723 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 3989
I20260805 22:42:47.452842 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 4180
I20260805 22:42:47.461442 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 4369
I20260805 22:42:47.474610 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 3850
I20260805 22:42:47.487589 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:39127
--webserver_interface=127.23.197.126
--webserver_port=39633
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:39127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:42:47.576081 4556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:47.576337 4556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:47.576380 4556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:47.577590 4556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:42:47.577663 4556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:47.577699 4556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:47.577732 4556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:42:47.577760 4556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:42:47.579324 4556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:39127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:39127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=39633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:47.579634 4556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:47.579885 4556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:47.585059 4562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:47.585057 4563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:47.585299 4556 server_base.cc:1034] running on GCE node
W20260805 22:42:47.585052 4565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:47.585670 4556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:47.585990 4556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:47.587169 4556 hybrid_clock.cc:648] HybridClock initialized: now 1785969767587154 us; error 45 us; skew 500 ppm
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:47.595714 4556 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:42:47.596218 4556 webserver.cc:460] Webserver started at http://127.23.197.126:39633/ using document root <none> and password file <none>
I20260805 22:42:47.596467 4556 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:47.596530 4556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:47.598078 4556 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:47.598852 4582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.599041 4556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:47.599134 4556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "d2b07efbc39348fc98cb37d6b5554e84"
format_stamp: "Formatted at 2026-08-05 22:42:46 on dist-test-slave-g170"
server_key: "b16338a401e10656397db6b57340991f"
server_key_iv: "b1d0522d2bdb9fe3e5e08c6be8ae253e"
server_key_version: "encryptionkey@0"
I20260805 22:42:47.599488 4556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:47.616802 4556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:47.618950 4556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:47.619071 4556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:47.623378 4556 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:39127
I20260805 22:42:47.623438 4659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:39127 every 8 connection(s)
I20260805 22:42:47.623821 4556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:42:47.625483 4660 sys_catalog.cc:263] Verifying existing consensus state
I20260805 22:42:47.626138 4660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: Bootstrap starting.
I20260805 22:42:47.633914 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 4556
I20260805 22:42:47.634312 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:35245
--local_ip_for_outbound_sockets=127.23.197.65
--tserver_master_addrs=127.23.197.126:39127
--webserver_port=43713
--webserver_interface=127.23.197.65
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.197.65
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:42:47.636041 4660 log.cc:824] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: Log is configured to *not* fsync() on all Append() calls
I20260805 22:42:47.638057 4660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:42:47.638397 4660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: Bootstrap complete.
I20260805 22:42:47.640877 4660 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } }
I20260805 22:42:47.641129 4660 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b07efbc39348fc98cb37d6b5554e84, State: Initialized, Role: FOLLOWER
I20260805 22:42:47.641317 4660 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } }
I20260805 22:42:47.641382 4660 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:42:47.641444 4660 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:42:47.641492 4660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 1 FOLLOWER]: Advancing to term 2
I20260805 22:42:47.642514 4660 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } }
I20260805 22:42:47.642606 4660 leader_election.cc:302] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2b07efbc39348fc98cb37d6b5554e84; no voters:
I20260805 22:42:47.642836 4660 leader_election.cc:288] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 22:42:47.642874 4665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 2 FOLLOWER]: Leader election won for term 2
I20260805 22:42:47.643004 4665 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [term 2 LEADER]: Becoming Leader. State: Replica: d2b07efbc39348fc98cb37d6b5554e84, State: Running, Role: LEADER
I20260805 22:42:47.643129 4660 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:42:47.643203 4665 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } }
I20260805 22:42:47.643497 4665 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2b07efbc39348fc98cb37d6b5554e84. Latest consensus state: current_term: 2 leader_uuid: "d2b07efbc39348fc98cb37d6b5554e84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } } }
I20260805 22:42:47.643589 4665 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:47.643505 4668 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d2b07efbc39348fc98cb37d6b5554e84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b07efbc39348fc98cb37d6b5554e84" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39127 } } }
I20260805 22:42:47.643784 4673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:42:47.643795 4668 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84 [sys.catalog]: This master's current role is: LEADER
I20260805 22:42:47.644347 4673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:42:47.644518 4673 catalog_manager.cc:1259] Loaded cluster ID: eb51fdeea5c341538d921aefbbb8046c
I20260805 22:42:47.644599 4673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:42:47.646212 4673 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:42:47.646389 4673 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d2b07efbc39348fc98cb37d6b5554e84: Loaded TSK: 0
I20260805 22:42:47.646586 4673 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 22:42:47.720587 4662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:47.720856 4662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:47.720902 4662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:47.722066 4662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:47.722131 4662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:47.722173 4662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:42:47.723603 4662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.65.keytab
--principal=oryx/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:35245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=43713
--tserver_master_addrs=127.23.197.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:47.723873 4662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:47.724107 4662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:47.729394 4696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:47.729390 4694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:47.729388 4701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:47.730189 4662 server_base.cc:1034] running on GCE node
I20260805 22:42:47.730436 4662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:47.730710 4662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:47.731930 4662 hybrid_clock.cc:648] HybridClock initialized: now 1785969767731912 us; error 43 us; skew 500 ppm
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969767, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:47.735131 4662 init.cc:377] Logged in from keytab as oryx/127.23.197.65@KRBTEST.COM (short username oryx)
I20260805 22:42:47.735744 4662 webserver.cc:460] Webserver started at http://127.23.197.65:43713/ using document root <none> and password file <none>
I20260805 22:42:47.736001 4662 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:47.736100 4662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:47.737824 4662 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:47.738631 4710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.738876 4662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:47.739001 4662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "e05a8a9482bf4f4c93325450417751bc"
format_stamp: "Formatted at 2026-08-05 22:42:46 on dist-test-slave-g170"
server_key: "bf7f5c6ec4abcea1ea2629752bc16195"
server_key_iv: "c2eda84155464370fb1f88997bc91b1b"
server_key_version: "encryptionkey@0"
I20260805 22:42:47.739403 4662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:47.761828 4662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:47.764210 4662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:47.764389 4662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:47.764667 4662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:47.765097 4662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:47.765156 4662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.765199 4662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:47.765228 4662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.771978 4662 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:35245
I20260805 22:42:47.772025 4874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:35245 every 8 connection(s)
I20260805 22:42:47.772403 4662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:47.781045 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 4662
I20260805 22:42:47.781368 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:41729
--local_ip_for_outbound_sockets=127.23.197.66
--tserver_master_addrs=127.23.197.126:39127
--webserver_port=40389
--webserver_interface=127.23.197.66
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.197.66
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:47.782420 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.773145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:52173)
0805 22:42:47.773379 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:47.773430 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:47.774045 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:47.774666 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:47.774669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:47.775267 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:47.775457 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.775462 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.777132 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.777133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.777999 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.778001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.778781 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.778781 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.778889 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.779465 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:47.779475 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:47.782191 (+ 2716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":119,"threads_started":1}
W20260805 22:42:47.782473 4879 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:47.773276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:52173 (local address 127.23.197.126:39127)
0805 22:42:47.774250 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:47.774260 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:47.774295 (+ 35us) server_negotiation.cc:407] Connection header received
0805 22:42:47.774414 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:47.774420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:47.774470 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:47.774594 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:47.775575 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.777023 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.778093 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.778678 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.778789 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.782324 (+ 3535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:52173: BlockingRecv error: recv got EOF from 127.23.197.65:52173 (error 108)
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":82,"threads_started":1}
W20260805 22:42:47.783149 4876 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:39127 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.197.126:39127: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:47.788504 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.783451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:49535)
0805 22:42:47.783509 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:47.783525 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:47.783586 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:47.783744 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:47.783745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:47.783892 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:47.783942 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.783943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.784620 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.784621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.785505 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.785509 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.785936 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.785937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.786013 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.786495 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:47.786502 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:47.788430 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260805 22:42:47.788600 4879 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:47.783509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:49535 (local address 127.23.197.126:39127)
0805 22:42:47.783572 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:47.783575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:47.783585 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:47.783650 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:47.783651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:47.783663 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:47.783739 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:47.784035 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.784534 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.785614 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.785846 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.785918 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.788540 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:49535: BlockingRecv error: recv got EOF from 127.23.197.65:49535 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:47.794064 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.788905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:48875)
0805 22:42:47.788967 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:47.788981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:47.789034 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:47.789220 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:47.789221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:47.789402 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:47.789443 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.789445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.790131 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.790133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.791000 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.791004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.791393 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.791395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.791461 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.791916 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:47.791923 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:47.794010 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260805 22:42:47.794243 4876 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 22:42:47.794255 4879 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:47.789028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:48875 (local address 127.23.197.126:39127)
0805 22:42:47.789096 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:47.789098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:47.789105 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:42:47.789125 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:47.789125 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:47.789133 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:47.789193 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:47.789522 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.790010 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.791119 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.791304 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.791367 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.794171 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:48875: BlockingRecv error: recv got EOF from 127.23.197.65:48875 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260805 22:42:47.872822 4881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:47.873052 4881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:47.873080 4881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:47.874317 4881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:47.874377 4881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:47.874406 4881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:42:47.875929 4881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.66.keytab
--principal=oryx/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:41729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=40389
--tserver_master_addrs=127.23.197.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:47.876202 4881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:47.876471 4881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:47.881851 4891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:47.881922 4893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:47.881868 4889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:47.882369 4881 server_base.cc:1034] running on GCE node
I20260805 22:42:47.882588 4881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:47.882860 4881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:47.884028 4881 hybrid_clock.cc:648] HybridClock initialized: now 1785969767884000 us; error 34 us; skew 500 ppm
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969767, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:47.886745 4881 init.cc:377] Logged in from keytab as oryx/127.23.197.66@KRBTEST.COM (short username oryx)
I20260805 22:42:47.887210 4881 webserver.cc:460] Webserver started at http://127.23.197.66:40389/ using document root <none> and password file <none>
I20260805 22:42:47.887465 4881 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:47.887547 4881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:47.889066 4881 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:47.889806 4903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.890166 4881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:42:47.890286 4881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "35df061a072348c2b72838407578dd41"
format_stamp: "Formatted at 2026-08-05 22:42:47 on dist-test-slave-g170"
server_key: "3a21aaaa3f23b582e0534b477af3e695"
server_key_iv: "b7818a0add4878516e7f567a9aa7d352"
server_key_version: "encryptionkey@0"
I20260805 22:42:47.890637 4881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:47.898799 4881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:47.901250 4881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:47.901422 4881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:47.901721 4881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:47.902094 4881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:47.902155 4881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.902199 4881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:47.902225 4881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:47.908363 4881 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:41729
I20260805 22:42:47.908407 5081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:41729 every 8 connection(s)
I20260805 22:42:47.908865 4881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:47.916548 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.909596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:37221)
0805 22:42:47.909847 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:47.909884 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:47.910378 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:47.910595 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:47.910598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:47.911217 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:47.911414 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.911419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.912140 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.912141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.913053 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.913056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.913481 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.913482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.913600 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.914204 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:47.914213 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:47.916397 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":123,"threads_started":1}
W20260805 22:42:47.916574 4879 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:47.909634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:37221 (local address 127.23.197.126:39127)
0805 22:42:47.909725 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:47.909729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:47.909973 (+ 244us) server_negotiation.cc:407] Connection header received
0805 22:42:47.910464 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:47.910465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:47.910481 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:47.910541 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:47.911516 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.912022 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.913174 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.913392 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.913468 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.916525 (+ 3057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:37221: BlockingRecv error: recv got EOF from 127.23.197.66:37221 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260805 22:42:47.916894 5083 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:39127 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.197.126:39127: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
I20260805 22:42:47.918879 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 4881
I20260805 22:42:47.919207 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:32849
--local_ip_for_outbound_sockets=127.23.197.67
--tserver_master_addrs=127.23.197.126:39127
--webserver_port=36583
--webserver_interface=127.23.197.67
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.197.67
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:47.921790 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.917142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:48071)
0805 22:42:47.917196 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:47.917207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:47.917253 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:47.917404 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:47.917405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:47.917532 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:47.917579 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.917580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.918252 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.918254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.918763 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.918765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.919170 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.919171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.919246 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.919660 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:47.919665 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:47.921705 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260805 22:42:47.921864 4879 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:47.917193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:48071 (local address 127.23.197.126:39127)
0805 22:42:47.917242 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:47.917244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:47.917254 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:47.917318 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:47.917319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:47.917328 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:47.917401 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:47.917709 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.918173 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.918849 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.919094 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.919173 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.921797 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:48071: BlockingRecv error: recv got EOF from 127.23.197.66:48071 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:47 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:47.926927 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:47.922275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:49929)
0805 22:42:47.922320 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:47.922329 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:47.922388 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:47.922512 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:47.922513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:47.922640 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:47.922665 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.922667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.923293 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.923294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.923784 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:47.923785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.924211 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.924212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:47.924274 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.924718 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:47.924723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:47.926842 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260805 22:42:47.926982 4879 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:47.922344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:49929 (local address 127.23.197.126:39127)
0805 22:42:47.922408 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:47.922410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:47.922419 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:47.922440 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:47.922441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:47.922450 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:47.922509 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:47.922749 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.923212 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.923866 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:47.924130 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:47.924213 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:47.926916 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:49929: BlockingRecv error: recv got EOF from 127.23.197.66:49929 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260805 22:42:47.927116 5083 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 22:42:48.007301 5086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:42:48.007515 5086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:42:48.007540 5086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:42:48.008715 5086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:42:48.008814 5086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:42:48.008888 5086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:42:48.010437 5086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/krb5kdc/oryx_127.23.197.67.keytab
--principal=oryx/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:32849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=36583
--tserver_master_addrs=127.23.197.126:39127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:42:48.010727 5086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:42:48.010982 5086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:42:48.016301 5093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:42:48.016269 5095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:48.016450 5086 server_base.cc:1034] running on GCE node
W20260805 22:42:48.016291 5098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:42:48.016831 5086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:42:48.017059 5086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:42:48.018208 5086 hybrid_clock.cc:648] HybridClock initialized: now 1785969768018196 us; error 29 us; skew 500 ppm
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969768, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:42:48.020745 5086 init.cc:377] Logged in from keytab as oryx/127.23.197.67@KRBTEST.COM (short username oryx)
I20260805 22:42:48.021196 5086 webserver.cc:460] Webserver started at http://127.23.197.67:36583/ using document root <none> and password file <none>
I20260805 22:42:48.021401 5086 fs_manager.cc:359] Metadata directory not provided
I20260805 22:42:48.021456 5086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:42:48.022969 5086 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:42:48.023593 5107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:48.023816 5086 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:42:48.023893 5086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "adba40d0cd04430a9622fac9876018e4"
format_stamp: "Formatted at 2026-08-05 22:42:47 on dist-test-slave-g170"
server_key: "53e3d22a7ec6c6a5561e4becf77bdd11"
server_key_iv: "99dac8b1e1b7f11a86cbe55311f5b1a5"
server_key_version: "encryptionkey@0"
I20260805 22:42:48.024212 5086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:42:48.030762 5086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:42:48.033094 5086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:42:48.033205 5086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:42:48.033375 5086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:42:48.033689 5086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:42:48.033730 5086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:48.033773 5086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:42:48.033799 5086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:42:48.039762 5086 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:32849
I20260805 22:42:48.039849 5279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:32849 every 8 connection(s)
I20260805 22:42:48.040177 5086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:42:48.046204 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 5086
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
W20260805 22:42:48.049232 4879 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:48.041092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:53323 (local address 127.23.197.126:39127)
0805 22:42:48.041143 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:48.041145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:48.041398 (+ 253us) server_negotiation.cc:407] Connection header received
0805 22:42:48.042065 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:48.042066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:48.042079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:48.042144 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:48.043153 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.043725 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.044855 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.045063 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.045150 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.049187 (+ 4037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:53323: BlockingRecv error: recv got EOF from 127.23.197.67:53323 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:42:48.049314 5283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:48.041079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:53323)
0805 22:42:48.041278 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:48.041314 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:48.041940 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:48.042225 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:48.042229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:48.042866 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:48.043058 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.043063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.043836 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.043837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.044738 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.044740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.045163 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.045164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.045296 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.045817 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:48.045826 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:48.049107 (+ 3281us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
W20260805 22:42:48.049539 5281 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:39127 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.197.126:39127: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:48.053835 5283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:48.049761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:41747)
0805 22:42:48.049812 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:48.049822 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:48.049860 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:48.049981 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:48.049981 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:48.050105 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:48.050145 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.050146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.050757 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.050758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.051254 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.051255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.051607 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.051608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.051682 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.052054 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:48.052058 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:48.053751 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260805 22:42:48.053887 4879 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:48.049815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:41747 (local address 127.23.197.126:39127)
0805 22:42:48.049862 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:48.049864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:48.049872 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:48.049913 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:48.049914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:48.049924 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:48.049967 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:48.050232 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.050672 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.051346 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.051537 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.051608 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.053822 (+ 2214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:41747: BlockingRecv error: recv got EOF from 127.23.197.67:41747 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260805 22:42:48.054486 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:48.046845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50728 (local address 127.23.197.126:39127)
0805 22:42:48.047011 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:48.047013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:48.047021 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:42:48.047042 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:48.047043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:48.047055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:48.047124 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:48.047770 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.048224 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.049442 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.049730 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.049814 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.051128 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:42:48.051132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:42:48.051132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:42:48.051151 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:42:48.053019 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:48.053394 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:48.053397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:48.053398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:48.053496 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:42:48.053717 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:42:48.053718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:42:48.053718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:42:48.053937 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:42:48.054087 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0805 22:42:48.054212 (+ 125us) server_negotiation.cc:299] Negotiation successful
0805 22:42:48.054323 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:48.058434 5283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:48.054331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:57917)
0805 22:42:48.054374 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:48.054384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:48.054425 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:48.054722 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:48.054723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:48.054827 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:48.054852 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.054853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.055588 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.055589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.056061 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.056063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.056428 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.056429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.056485 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.056843 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:48.056847 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:48.058359 (+ 1512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260805 22:42:48.058480 4879 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:48.054375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:57917 (local address 127.23.197.126:39127)
0805 22:42:48.054430 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:48.054433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:48.054457 (+ 24us) server_negotiation.cc:407] Connection header received
0805 22:42:48.054475 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:48.054476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:48.054484 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:48.054528 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:48.054939 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.055514 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.056152 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.056348 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.056416 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.058422 (+ 2006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:57917: BlockingRecv error: recv got EOF from 127.23.197.67:57917 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260805 22:42:48.058600 5281 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:48.800007 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:48.794758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:60751)
0805 22:42:48.794947 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:48.794960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:48.795029 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:48.795187 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:48.795188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:48.795376 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:48.795428 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.795430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.796146 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.796147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.796730 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.796732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.797196 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.797197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.797243 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.797727 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:48.797882 (+ 155us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:48.799921 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":97,"threads_started":1}
W20260805 22:42:48.800170 5290 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:48.794831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:60751 (local address 127.23.197.126:39127)
0805 22:42:48.795000 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:48.795003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:48.795016 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:42:48.795078 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:48.795078 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:48.795089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:48.795152 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:48.795530 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.796057 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.796895 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.797100 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.797161 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.800063 (+ 2902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:60751: BlockingRecv error: recv got EOF from 127.23.197.65:60751 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:48 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:48.932386 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:48.927538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:59833)
0805 22:42:48.927724 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:48.927743 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:48.927808 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:48.928076 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:48.928077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:48.928285 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:48.928336 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.928338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.929071 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.929072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.929616 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:48.929618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.929962 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.929963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:48.930002 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.930510 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:48.930514 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:48.932315 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":137,"threads_started":1}
W20260805 22:42:48.932550 5290 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:48.927651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:59833 (local address 127.23.197.126:39127)
0805 22:42:48.927706 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:48.927708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:48.927854 (+ 146us) server_negotiation.cc:407] Connection header received
0805 22:42:48.927890 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:48.927891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:48.927906 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:48.928013 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:48.928424 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.928999 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.929700 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:48.929893 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:48.929951 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:48.932458 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:59833: BlockingRecv error: recv got EOF from 127.23.197.66:59833 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 05 22:42:49 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:49 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:49.064002 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:49.059032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:54171)
0805 22:42:49.059191 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:49.059205 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:49.059306 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:49.059482 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:49.059483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:49.059685 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:49.059735 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:49.059737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.060438 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.060439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:49.061069 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:49.061071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.061457 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.061458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:49.061527 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:49.062005 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:49.062010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:49.063891 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
W20260805 22:42:49.064043 5290 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:49.059148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:54171 (local address 127.23.197.126:39127)
0805 22:42:49.059195 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:49.059197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:49.059284 (+ 87us) server_negotiation.cc:407] Connection header received
0805 22:42:49.059377 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:49.059378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:49.059394 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:49.059474 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:49.059823 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.060338 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.061166 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.061365 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.061456 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:49.063977 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:54171: BlockingRecv error: recv got EOF from 127.23.197.67:54171 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 05 22:42:49 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:49 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:49.806186 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:49.800602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:35797)
0805 22:42:49.800797 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:49.800810 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:49.800875 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:49.801099 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:49.801101 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:49.801302 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:49.801354 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:49.801356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.802091 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.802093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:49.802636 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:49.802637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.803063 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.803064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:49.803145 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:49.803633 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:49.803638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:49.806124 (+ 2486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
W20260805 22:42:49.806394 5295 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:49.800748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:35797 (local address 127.23.197.126:39127)
0805 22:42:49.800930 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:49.800933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:49.800951 (+ 18us) server_negotiation.cc:407] Connection header received
0805 22:42:49.800976 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:49.800976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:49.800988 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:49.801060 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:49.801432 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.802009 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.802736 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.802981 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.803054 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:49.806317 (+ 3263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:35797: BlockingRecv error: recv got EOF from 127.23.197.65:35797 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":136,"threads_started":1}
Aug 05 22:42:49 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:49 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:49.938169 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:49.932968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:39443)
0805 22:42:49.933149 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:49.933164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:49.933273 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:49.933443 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:49.933445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:49.933645 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:49.933695 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:49.933697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.934451 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.934452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:49.935040 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:49.935042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.935456 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.935457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:49.935521 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:49.936014 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:49.936020 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:49.938060 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
W20260805 22:42:49.938244 5295 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:49.933076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:39443 (local address 127.23.197.126:39127)
0805 22:42:49.933153 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:49.933156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:49.933251 (+ 95us) server_negotiation.cc:407] Connection header received
0805 22:42:49.933340 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:49.933341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:49.933363 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:49.933437 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:49.933792 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.934348 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.935133 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:49.935358 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:49.935456 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:49.938160 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:39443: BlockingRecv error: recv got EOF from 127.23.197.66:39443 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 05 22:42:50 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:50 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:50.069547 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:50.064548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:54451)
0805 22:42:50.064809 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:50.064824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:50.064896 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:50.065074 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:50.065075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:50.065296 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:50.065346 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:50.065348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.066036 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.066037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:50.066577 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:50.066578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.066942 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.066943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:50.067003 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:50.067504 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:50.067510 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:50.069445 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":149,"threads_started":1}
W20260805 22:42:50.069602 5295 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:50.064725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:54451 (local address 127.23.197.126:39127)
0805 22:42:50.064818 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:50.064821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:50.064876 (+ 55us) server_negotiation.cc:407] Connection header received
0805 22:42:50.064967 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:50.064968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:50.064988 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:50.065049 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:50.065434 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.065941 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.066665 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.066863 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.066944 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:50.069534 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:54451: BlockingRecv error: recv got EOF from 127.23.197.67:54451 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 05 22:42:50 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:50 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:50.811805 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:50.806732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:35367)
0805 22:42:50.806919 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:50.806932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:50.806993 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:50.807245 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:50.807246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:50.807424 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:50.807476 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:50.807477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.808233 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.808235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:50.808904 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:50.808907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.809300 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.809301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:50.809369 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:50.809853 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:50.809858 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:50.811731 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
W20260805 22:42:50.811977 5299 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:50.806837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:35367 (local address 127.23.197.126:39127)
0805 22:42:50.806975 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:50.806977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:50.806987 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:50.807103 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:50.807104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:50.807119 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:50.807186 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:50.807587 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.808138 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.808994 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.809200 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.809261 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:50.811864 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:35367: BlockingRecv error: recv got EOF from 127.23.197.65:35367 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
Aug 05 22:42:50 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:50 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:50.944037 5300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:50.938742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:38767)
0805 22:42:50.938929 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:50.938943 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:50.939013 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:50.939248 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:50.939250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:50.939429 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:50.939478 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:50.939480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.940155 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.940156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:50.940705 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:50.940707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.941137 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.941138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:50.941191 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:50.941769 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:50.941773 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:50.943959 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":69,"threads_started":1}
W20260805 22:42:50.944242 5299 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:50.938975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:38767 (local address 127.23.197.126:39127)
0805 22:42:50.939074 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:50.939076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:50.939085 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:50.939117 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:50.939118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:50.939128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:50.939188 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:50.939559 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.940078 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.940831 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:50.941041 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:50.941099 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:50.944098 (+ 2999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:38767: BlockingRecv error: recv got EOF from 127.23.197.66:38767 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 05 22:42:51 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:51 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:51.074955 5301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:51.070176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:60839)
0805 22:42:51.070337 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:51.070347 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:51.070402 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:51.070515 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:51.070515 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:51.070700 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:51.070754 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:51.070757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.071403 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.071404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:51.071964 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:51.071966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.072317 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.072318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:51.072366 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:51.072896 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:51.072902 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:51.074872 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
W20260805 22:42:51.075037 5299 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:51.070241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:60839 (local address 127.23.197.126:39127)
0805 22:42:51.070314 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:51.070317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:51.070405 (+ 88us) server_negotiation.cc:407] Connection header received
0805 22:42:51.070442 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:51.070443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:51.070453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:51.070501 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:51.070817 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.071309 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.072046 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.072239 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.072293 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:51.074957 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:60839: BlockingRecv error: recv got EOF from 127.23.197.67:60839 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 05 22:42:51 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:51 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:51.817967 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:51.812435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:43191)
0805 22:42:51.812603 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:51.812614 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:51.812669 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:51.812913 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:51.812914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:51.813090 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:51.813142 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:51.813143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.813914 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.813916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:51.814464 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:51.814465 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.814999 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.815000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:51.815078 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:51.815629 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:51.815635 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:51.817906 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
W20260805 22:42:51.818130 5304 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:51.812500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:43191 (local address 127.23.197.126:39127)
0805 22:42:51.812701 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:51.812704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:51.812714 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:51.812738 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:51.812739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:51.812749 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:51.812850 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:51.813238 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.813826 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.814602 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.814880 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.814983 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:51.818033 (+ 3050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:43191: BlockingRecv error: recv got EOF from 127.23.197.65:43191 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":135,"threads_started":1}
Aug 05 22:42:51 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:51 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:51.950158 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:51.944648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:50127)
0805 22:42:51.944895 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:51.944906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:51.944976 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:51.945117 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:51.945118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:51.945291 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:51.945342 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:51.945344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.946030 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.946031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:51.946577 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:51.946579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.946989 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.946990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:51.947045 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:51.947572 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:51.947577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:51.950085 (+ 2508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":181,"threads_started":1}
W20260805 22:42:51.950320 5304 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:51.944683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:50127 (local address 127.23.197.126:39127)
0805 22:42:51.944812 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:51.944815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:51.944992 (+ 177us) server_negotiation.cc:407] Connection header received
0805 22:42:51.945028 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:51.945029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:51.945040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:51.945115 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:51.945437 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.945952 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.946676 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:51.946872 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:51.946939 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:51.950230 (+ 3291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:50127: BlockingRecv error: recv got EOF from 127.23.197.66:50127 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 05 22:42:52 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:52 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:52.080791 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:52.075516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:43251)
0805 22:42:52.075701 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:52.075714 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:52.075780 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:52.076019 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:52.076021 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:52.076227 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:52.076279 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:52.076281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.077050 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.077051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:52.077638 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:52.077639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.077999 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.078000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:52.078053 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:52.078586 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:52.078592 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:52.080645 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":112,"threads_started":1}
W20260805 22:42:52.080835 5304 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:52.075651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:43251 (local address 127.23.197.126:39127)
0805 22:42:52.075749 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:52.075751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:52.075768 (+ 17us) server_negotiation.cc:407] Connection header received
0805 22:42:52.075888 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:52.075890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:52.075908 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:52.075974 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:52.076353 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.076966 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.077725 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.077924 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.077982 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:52.080733 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:43251: BlockingRecv error: recv got EOF from 127.23.197.67:43251 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 05 22:42:52 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:52 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:52.827345 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:52.818481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:38215)
0805 22:42:52.818705 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:52.818717 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:52.818785 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:52.818925 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:52.818926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:52.819102 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:52.819154 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:52.819156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.819830 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.819831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:52.820359 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:52.820361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.820848 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.820850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:52.820919 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:52.825187 (+ 4268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:52.825194 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:52.827282 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":117,"threads_started":1}
W20260805 22:42:52.827483 5312 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:52.818594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:38215 (local address 127.23.197.126:39127)
0805 22:42:52.818707 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:52.818709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:52.818788 (+ 79us) server_negotiation.cc:407] Connection header received
0805 22:42:52.818833 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:52.818833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:52.818845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:52.818905 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:52.819216 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.819726 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.820432 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.820714 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.820843 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:52.827391 (+ 6548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:38215: BlockingRecv error: recv got EOF from 127.23.197.65:38215 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":52,"threads_started":1}
Aug 05 22:42:52 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:52 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:52.955507 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:52.950689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:55109)
0805 22:42:52.950855 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:52.950884 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:52.950953 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:52.951140 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:52.951141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:52.951334 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:52.951383 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:52.951385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.952089 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.952090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:52.952622 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:52.952624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.953052 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.953053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:52.953113 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:52.953578 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:52.953583 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:52.955444 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
W20260805 22:42:52.955629 5312 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:52.950844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:55109 (local address 127.23.197.126:39127)
0805 22:42:52.950919 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:52.950921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:52.950935 (+ 14us) server_negotiation.cc:407] Connection header received
0805 22:42:52.951038 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:52.951039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:52.951053 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:52.951130 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:52.951470 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.951999 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.952737 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:52.952973 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:52.953054 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:52.955551 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:55109: BlockingRecv error: recv got EOF from 127.23.197.66:55109 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 05 22:42:53 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:53 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:53.086357 5315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:53.081387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:36917)
0805 22:42:53.081564 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:53.081577 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:53.081639 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:53.081788 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:53.081789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:53.081962 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:53.082016 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:53.082026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.082661 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.082662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:53.083248 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:53.083250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.083728 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.083729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:53.083814 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:53.084344 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:53.084349 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:53.086298 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
W20260805 22:42:53.086498 5312 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:53.081526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:36917 (local address 127.23.197.126:39127)
0805 22:42:53.081606 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:53.081608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:53.081627 (+ 19us) server_negotiation.cc:407] Connection header received
0805 22:42:53.081711 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:53.081712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:53.081724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:53.081769 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:53.082089 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.082581 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.083383 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.083660 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.083748 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:53.086406 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:36917: BlockingRecv error: recv got EOF from 127.23.197.67:36917 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 05 22:42:53 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:53 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:53.834610 5319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:53.827906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:57089)
0805 22:42:53.828064 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:53.828083 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:53.828140 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:53.828300 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:53.828301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:53.828481 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:53.828534 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:53.828535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.829327 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.829328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:53.829861 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:53.829862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.830288 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.830289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:53.830330 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:53.830810 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:53.830815 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:53.834547 (+ 3732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
W20260805 22:42:53.834780 5320 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:53.828021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:57089 (local address 127.23.197.126:39127)
0805 22:42:53.828159 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:53.828161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:53.828170 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:53.828195 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:53.828195 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:53.828206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:53.828278 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:53.828644 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.829245 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.829998 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.830211 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.830277 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:53.834681 (+ 4404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:57089: BlockingRecv error: recv got EOF from 127.23.197.65:57089 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
Aug 05 22:42:53 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:53 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:53.961529 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:53.956097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:36141)
0805 22:42:53.956338 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:53.956356 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:53.956462 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:53.956646 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:53.956648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:53.956923 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:53.956976 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:53.956979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.957860 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.957861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:53.958415 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:53.958416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.958812 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.958813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:53.958874 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:53.959413 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:53.959419 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:53.961456 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":111,"threads_started":1}
W20260805 22:42:53.961609 5320 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:53.956266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:36141 (local address 127.23.197.126:39127)
0805 22:42:53.956360 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:53.956363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:53.956439 (+ 76us) server_negotiation.cc:407] Connection header received
0805 22:42:53.956546 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:53.956547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:53.956560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:53.956641 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:53.957070 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.957759 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.958506 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:53.958731 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:53.958814 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:53.961561 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:36141: BlockingRecv error: recv got EOF from 127.23.197.66:36141 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 05 22:42:54 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:54 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:54.092662 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:54.086967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:54661)
0805 22:42:54.087115 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:54.087134 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:54.087200 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:54.087339 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:54.087340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:54.087509 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:54.087560 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:54.087561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.088394 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.088395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:54.089227 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:54.089231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.089658 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.089720 (+ 62us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:54.089795 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:54.090473 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:54.090479 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:54.092558 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
W20260805 22:42:54.092741 5320 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:54.087049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:54661 (local address 127.23.197.126:39127)
0805 22:42:54.087121 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:54.087123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:54.087183 (+ 60us) server_negotiation.cc:407] Connection header received
0805 22:42:54.087247 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:54.087248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:54.087260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:54.087320 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:54.087682 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.088276 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.089328 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.089550 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.089624 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:54.092646 (+ 3022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:54661: BlockingRecv error: recv got EOF from 127.23.197.67:54661 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 05 22:42:54 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:54 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:54.840682 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:54.835275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:42295)
0805 22:42:54.835462 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:54.835475 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:54.835544 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:54.835750 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:54.835752 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:54.835939 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:54.835992 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:54.835993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.836690 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.836691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:54.837363 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:54.837366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.837761 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.837762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:54.837828 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:54.838440 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:54.838446 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:54.840581 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":92,"threads_started":1}
W20260805 22:42:54.840866 5327 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:54.835390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:42295 (local address 127.23.197.126:39127)
0805 22:42:54.835514 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:54.835515 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:54.835524 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:54.835615 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:54.835616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:54.835627 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:54.835689 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:54.836073 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.836589 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.837474 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.837677 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.837740 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:54.840678 (+ 2938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:42295: BlockingRecv error: recv got EOF from 127.23.197.65:42295 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
Aug 05 22:42:54 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:54 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:54.967854 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:54.962186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:35793)
0805 22:42:54.962355 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:54.962367 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:54.962444 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:54.962720 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:54.962722 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:54.963004 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:54.963080 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:54.963083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.964069 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.964070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:54.964616 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:54.964619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.965100 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.965101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:54.965155 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:54.965705 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:54.965710 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:54.967787 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
W20260805 22:42:54.968020 5327 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:54.962301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:35793 (local address 127.23.197.126:39127)
0805 22:42:54.962398 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:54.962400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:54.962413 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:42:54.962520 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:54.962521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:54.962554 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:54.962646 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:54.963188 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.963953 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.964716 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:54.965004 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:54.965071 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:54.967929 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:35793: BlockingRecv error: recv got EOF from 127.23.197.66:35793 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 05 22:42:55 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:55 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:55.098654 5331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:55.093621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:37551)
0805 22:42:55.093812 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:55.093822 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:55.093889 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:55.094102 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:55.094103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:55.094338 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:55.094401 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:55.094404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.095076 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.095078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:55.095621 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:55.095623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.095985 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.095986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:55.096046 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:55.096576 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:55.096581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:55.098590 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":78,"threads_started":1}
W20260805 22:42:55.098853 5327 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:55.093664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:37551 (local address 127.23.197.126:39127)
0805 22:42:55.093744 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:55.093746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:55.093880 (+ 134us) server_negotiation.cc:407] Connection header received
0805 22:42:55.093994 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:55.093995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:55.094015 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:55.094082 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:55.094479 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.094981 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.095706 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.095906 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.095985 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:55.098760 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:37551: BlockingRecv error: recv got EOF from 127.23.197.67:37551 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 05 22:42:55 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:55 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:55.846786 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:55.841248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:43895)
0805 22:42:55.841402 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:55.841413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:55.841465 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:55.841730 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:55.841730 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:55.841909 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:55.841963 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:55.841965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.842790 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.842791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:55.843428 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:55.843430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.843888 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.843889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:55.843932 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:55.844567 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:55.844571 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:55.846684 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
W20260805 22:42:55.846925 5334 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:55.841395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:43895 (local address 127.23.197.126:39127)
0805 22:42:55.841551 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:55.841553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:55.841564 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:55.841598 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:55.841599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:55.841613 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:55.841688 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:55.842038 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.842708 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.843562 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.843819 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.843879 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:55.846830 (+ 2951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:43895: BlockingRecv error: recv got EOF from 127.23.197.65:43895 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
Aug 05 22:42:55 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:55 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:55.974278 5336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:55.968503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:36303)
0805 22:42:55.968752 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:55.968852 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:55.968975 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:55.969170 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:55.969172 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:55.969379 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:55.969428 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:55.969431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.970378 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.970379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:55.971135 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:55.971138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.971515 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.971516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:55.971574 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:55.972087 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:55.972092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:55.974173 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":115,"threads_started":1}
W20260805 22:42:55.974337 5334 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:55.968662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:36303 (local address 127.23.197.126:39127)
0805 22:42:55.968853 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:55.968855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:55.968952 (+ 97us) server_negotiation.cc:407] Connection header received
0805 22:42:55.969067 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:55.969068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:55.969089 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:55.969161 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:55.969521 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.970286 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.971229 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:55.971437 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:55.971518 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:55.974269 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:36303: BlockingRecv error: recv got EOF from 127.23.197.66:36303 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
Aug 05 22:42:56 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:56 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:56.105043 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:56.099259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:38625)
0805 22:42:56.099488 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:56.099499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:56.099563 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:56.099723 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:56.099724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:56.099911 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:56.099963 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:56.099965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.100629 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.100630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:56.101249 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:56.101251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.101582 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.101583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:56.101666 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:56.102257 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:56.102263 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:56.104977 (+ 2714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":125,"threads_started":1}
W20260805 22:42:56.105113 5334 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:56.099371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:38625 (local address 127.23.197.126:39127)
0805 22:42:56.099436 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:56.099438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:56.099561 (+ 123us) server_negotiation.cc:407] Connection header received
0805 22:42:56.099616 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:56.099616 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:56.099632 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:56.099691 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:56.100031 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.100521 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.101329 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.101512 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.101570 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:56.105061 (+ 3491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:38625: BlockingRecv error: recv got EOF from 127.23.197.67:38625 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 05 22:42:56 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:56 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:56.852582 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:56.847277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:59165)
0805 22:42:56.847478 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:56.847497 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:56.847558 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:56.847779 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:56.847781 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:56.847959 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:56.848015 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:56.848016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.848855 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.848856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:56.849487 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:56.849489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.849916 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.849917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:56.849966 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:56.850499 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:56.850504 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:56.852512 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":122,"threads_started":1}
W20260805 22:42:56.852701 5344 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:56.847387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:59165 (local address 127.23.197.126:39127)
0805 22:42:56.847534 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:56.847536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:56.847546 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:42:56.847626 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:56.847626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:56.847637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:56.847700 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:56.848101 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.848605 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.849600 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.849830 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.849902 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:56.852619 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:59165: BlockingRecv error: recv got EOF from 127.23.197.65:59165 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":59,"threads_started":1}
Aug 05 22:42:56 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:56 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:56.980983 5345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:56.974973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:41141)
0805 22:42:56.975145 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:56.975159 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:56.975222 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:56.975382 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:56.975383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:56.975574 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:56.975628 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:56.975629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.976614 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.976615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:56.977573 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:56.977576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.977992 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.977993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:56.978039 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:56.978612 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:56.978618 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:56.980908 (+ 2290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
W20260805 22:42:56.981063 5344 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:56.975138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:41141 (local address 127.23.197.126:39127)
0805 22:42:56.975191 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:56.975198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:56.975209 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:42:56.975277 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:56.975278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:56.975290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:56.975356 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:56.975769 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.976362 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.977704 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:56.977911 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:56.977974 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:56.981014 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:41141: BlockingRecv error: recv got EOF from 127.23.197.66:41141 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 05 22:42:57 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:57 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:57.112077 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:57.105624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:51947)
0805 22:42:57.106862 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:57.106874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:57.106973 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:57.107146 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:57.107147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:57.107383 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:57.107439 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:57.107441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.108187 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.108188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:57.108856 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:57.108858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.109225 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.109226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:57.109300 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:57.109798 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:57.109803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:57.112014 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1217,"thread_start_us":1160,"threads_started":1}
W20260805 22:42:57.112236 5344 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:57.105712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:51947 (local address 127.23.197.126:39127)
0805 22:42:57.105773 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:57.105775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:57.106961 (+ 1186us) server_negotiation.cc:407] Connection header received
0805 22:42:57.107036 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:57.107037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:57.107053 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:57.107140 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:57.107529 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.108094 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.108942 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.109139 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.109219 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:57.112132 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:51947: BlockingRecv error: recv got EOF from 127.23.197.67:51947 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 05 22:42:57 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:57 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:57.863111 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:57.853235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:36359)
0805 22:42:57.853395 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:57.853408 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:57.853475 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:57.853647 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:57.853648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:57.853854 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:57.853904 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:57.853906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.854619 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.854621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:57.855155 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:57.855157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.855546 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.855547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:57.855616 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:57.861000 (+ 5384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:57.861007 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:57.863046 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
W20260805 22:42:57.863305 5350 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:57.853297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:36359 (local address 127.23.197.126:39127)
0805 22:42:57.853438 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:57.853440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:57.853455 (+ 15us) server_negotiation.cc:407] Connection header received
0805 22:42:57.853550 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:57.853551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:57.853565 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:57.853634 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:57.853994 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.854522 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.855261 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.855468 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.855547 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:57.863218 (+ 7671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:36359: BlockingRecv error: recv got EOF from 127.23.197.65:36359 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
Aug 05 22:42:57 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:57 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:57.986732 5351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:57.981661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:33211)
0805 22:42:57.981933 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:57.981945 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:57.982019 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:57.982198 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:57.982200 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:57.982387 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:57.982439 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:57.982441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.983221 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.983222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:57.983791 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:57.983793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.984198 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.984199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:57.984244 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:57.984852 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:57.984862 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:57.986663 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":188,"threads_started":1}
W20260805 22:42:57.986937 5350 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:57.981800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:33211 (local address 127.23.197.126:39127)
0805 22:42:57.981905 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:57.981908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:57.982025 (+ 117us) server_negotiation.cc:407] Connection header received
0805 22:42:57.982073 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:57.982074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:57.982087 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:57.982146 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:57.982560 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.983125 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.983889 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:57.984116 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:57.984187 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:57.986863 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:33211: BlockingRecv error: recv got EOF from 127.23.197.66:33211 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 05 22:42:58 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:58 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:58.117468 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:58.112620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:49499)
0805 22:42:58.112815 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:58.112827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:58.112885 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:58.113034 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:58.113035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:58.113218 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:58.113267 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:58.113268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.113975 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.113976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:58.114522 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:58.114524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.114899 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.114899 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:58.114943 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:58.115476 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:58.115482 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:58.117409 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":113,"threads_started":1}
W20260805 22:42:58.117749 5350 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:58.112845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:49499 (local address 127.23.197.126:39127)
0805 22:42:58.112922 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:58.112924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:58.112933 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:58.112955 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:58.112955 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:58.112968 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:58.113029 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:58.113339 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.113892 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.114608 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.114825 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.114885 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:58.117664 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:49499: BlockingRecv error: recv got EOF from 127.23.197.67:49499 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 05 22:42:58 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:58 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:58.868973 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:58.863680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:54447)
0805 22:42:58.863846 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:58.863858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:58.863913 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:58.864142 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:58.864144 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:58.864334 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:58.864386 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:58.864387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.865145 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.865147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:58.865684 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:58.865686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.866040 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.866041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:58.866085 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:58.866722 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:58.866728 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:58.868898 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
W20260805 22:42:58.869117 5355 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:58.863800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:54447 (local address 127.23.197.126:39127)
0805 22:42:58.863988 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:58.863990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:58.863999 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:42:58.864024 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:58.864024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:58.864034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:58.864107 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:58.864497 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.865049 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.865771 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.865968 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.866026 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:58.869018 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:54447: BlockingRecv error: recv got EOF from 127.23.197.65:54447 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
Aug 05 22:42:58 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:58 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:58.992357 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:58.987260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:34523)
0805 22:42:58.987443 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:58.987460 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:58.987550 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:58.987718 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:58.987719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:58.987917 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:58.987964 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:58.987965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.988643 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.988644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:58.989291 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:58.989294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.989666 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.989674 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:58.989735 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:58.990239 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:58.990244 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:58.992284 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":108,"threads_started":1}
W20260805 22:42:58.992468 5355 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:58.987324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:34523 (local address 127.23.197.126:39127)
0805 22:42:58.987408 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:58.987411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:58.987527 (+ 116us) server_negotiation.cc:407] Connection header received
0805 22:42:58.987629 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:58.987630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:58.987643 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:58.987708 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:58.988047 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.988554 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.989383 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:58.989589 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:58.989669 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:58.992392 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:34523: BlockingRecv error: recv got EOF from 127.23.197.66:34523 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 05 22:42:59 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:59 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:59.123538 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:59.118118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:56541)
0805 22:42:59.118364 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:59.118382 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:59.118486 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:59.118678 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:59.118679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:59.118899 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:59.118952 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:59.118954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.119749 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.119750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:59.120342 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:59.120344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.120808 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.120810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:59.120881 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:59.121386 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:59.121392 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:59.123459 (+ 2067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":139,"threads_started":1}
W20260805 22:42:59.123950 5355 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:59.118205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:56541 (local address 127.23.197.126:39127)
0805 22:42:59.118347 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:59.118350 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:59.118462 (+ 112us) server_negotiation.cc:407] Connection header received
0805 22:42:59.118567 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:59.118567 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:59.118592 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:59.118660 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:59.119050 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.119628 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.120445 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.120668 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.120793 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:59.123745 (+ 2952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:56541: BlockingRecv error: recv got EOF from 127.23.197.67:56541 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 05 22:42:59 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:59 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:59.874780 5359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:59.869636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:34735)
0805 22:42:59.869834 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:59.869846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:59.869907 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:59.870099 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:59.870100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:59.870315 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:59.870369 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:59.870371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.871025 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.871025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:59.871632 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:59.871654 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.872005 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.872006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:59.872057 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:59.872539 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:59.872544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:59.874708 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":114,"threads_started":1}
W20260805 22:42:59.874908 5360 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:59.869742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:34735 (local address 127.23.197.126:39127)
0805 22:42:59.869912 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:59.869914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:59.869964 (+ 50us) server_negotiation.cc:407] Connection header received
0805 22:42:59.869988 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:59.869989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:59.870002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:59.870075 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:59.870452 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.870949 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.871737 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.871932 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.871987 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:59.874816 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:34735: BlockingRecv error: recv got EOF from 127.23.197.65:34735 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
Aug 05 22:42:59 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:42:59 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:42:59.999006 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:42:59.993135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:37691)
0805 22:42:59.993311 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0805 22:42:59.993332 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 22:42:59.993388 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:42:59.993547 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:42:59.993548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:42:59.993740 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:42:59.993801 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:59.993803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.994685 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.994687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:59.995616 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:42:59.995618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.996011 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.996012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:42:59.996065 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:59.996661 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:42:59.996668 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:42:59.998913 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":116,"threads_started":1}
W20260805 22:42:59.999130 5360 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:42:59.993297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:37691 (local address 127.23.197.126:39127)
0805 22:42:59.993358 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0805 22:42:59.993360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:42:59.993375 (+ 15us) server_negotiation.cc:407] Connection header received
0805 22:42:59.993443 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:42:59.993444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:42:59.993456 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:42:59.993531 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:42:59.993919 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.994559 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.995701 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:42:59.995932 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:42:59.995994 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:42:59.999047 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:37691: BlockingRecv error: recv got EOF from 127.23.197.66:37691 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 05 22:43:00 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:43:00 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:43:00.129804 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:00.124161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:48053)
0805 22:43:00.124350 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:00.124362 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:00.124426 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:00.124621 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:00.124628 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:00.124862 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:00.124915 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:00.124917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:00.125662 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:00.125663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:00.126259 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:00.126261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:00.126765 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:00.126766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:00.126819 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:00.127353 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:00.127358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:00.129719 (+ 2361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":67,"threads_started":1}
W20260805 22:43:00.130038 5360 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:00.124385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:48053 (local address 127.23.197.126:39127)
0805 22:43:00.124479 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:00.124481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:00.124490 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:43:00.124515 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:00.124516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:00.124529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:00.124591 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:00.125024 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:00.125535 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:00.126416 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:00.126658 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:00.126734 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:00.129963 (+ 3229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:48053: BlockingRecv error: recv got EOF from 127.23.197.67:48053 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 05 22:43:00 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:43:00 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:43:00.880537 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:00.875398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:46573)
0805 22:43:00.875570 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:00.875581 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:00.875642 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:00.875896 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:00.875897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:00.876114 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:00.876176 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:00.876178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:00.876947 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:00.876948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:00.877498 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:00.877500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:00.877887 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:00.877888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:00.877936 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:00.878516 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:00.878521 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:00.880472 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
W20260805 22:43:00.880676 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:00.875552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:46573 (local address 127.23.197.126:39127)
0805 22:43:00.875718 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:00.875720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:00.875758 (+ 38us) server_negotiation.cc:407] Connection header received
0805 22:43:00.875785 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:00.875786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:00.875798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:00.875874 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:00.876259 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:00.876862 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:00.877580 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:00.877811 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:00.877866 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:00.880600 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:46573: BlockingRecv error: recv got EOF from 127.23.197.65:46573 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
Aug 05 22:43:00 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:43:01 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:43:01.005568 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:00.999722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:44719)
0805 22:43:00.999926 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:00.999943 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:01.000026 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:01.000222 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:01.000224 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:01.000433 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:01.000500 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:01.000503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.001379 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.001380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:01.002014 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:01.002017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.002528 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.002529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:01.002580 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:01.003297 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:01.003303 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:01.005496 (+ 2193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":104,"threads_started":1}
W20260805 22:43:01.005676 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:00.999870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:44719 (local address 127.23.197.126:39127)
0805 22:43:00.999945 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:00.999947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:01.000006 (+ 59us) server_negotiation.cc:407] Connection header received
0805 22:43:01.000097 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:01.000098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:01.000112 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:01.000161 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:01.000625 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.001273 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.002141 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.002440 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.002511 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:01.005608 (+ 3097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:44719: BlockingRecv error: recv got EOF from 127.23.197.66:44719 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 05 22:43:01 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:43:01 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:43:01.135645 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:01.130358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:37577)
0805 22:43:01.130547 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:01.130562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:01.130650 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:01.130807 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:01.130808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:01.131002 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:01.131055 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:01.131058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.131757 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.131758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:01.132333 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:01.132336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.132711 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.132712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:01.132839 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:01.133345 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:01.133351 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:01.135563 (+ 2212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":75,"threads_started":1}
W20260805 22:43:01.135821 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:01.130502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:37577 (local address 127.23.197.126:39127)
0805 22:43:01.130583 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:01.130585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:01.130651 (+ 66us) server_negotiation.cc:407] Connection header received
0805 22:43:01.130721 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:01.130722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:01.130734 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:01.130777 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:01.131168 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.131656 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.132425 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.132608 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.132671 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:01.135743 (+ 3072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:37577: BlockingRecv error: recv got EOF from 127.23.197.67:37577 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 05 22:43:01 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:43:01 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:43:01.886847 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:01.881120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:36263)
0805 22:43:01.881364 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:01.881380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:01.881470 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:01.881653 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:01.881654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:01.881853 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:01.881930 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:01.881932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.882644 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.882645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:01.883421 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:01.883424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.883879 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.883881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:01.883936 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:01.884528 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:01.884533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:01.886764 (+ 2231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":121,"threads_started":1}
W20260805 22:43:01.887038 5377 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:01.881281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:36263 (local address 127.23.197.126:39127)
0805 22:43:01.881443 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:01.881445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:01.881458 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:43:01.881547 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:01.881547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:01.881568 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:01.881627 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:01.882009 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.882558 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.883518 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:01.883778 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:01.883845 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:01.886910 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:36263: BlockingRecv error: recv got EOF from 127.23.197.65:36263 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
Aug 05 22:43:02 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:43:02 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:43:02.011588 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:02.006178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:38357)
0805 22:43:02.006362 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:02.006374 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:02.006437 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:02.006599 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:02.006600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:02.006817 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:02.006872 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:02.006874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.007546 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.007547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:02.008136 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:02.008138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.008520 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.008521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:02.008570 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:02.009185 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:02.009190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:02.011522 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
W20260805 22:43:02.011848 5377 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:02.006273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:38357 (local address 127.23.197.126:39127)
0805 22:43:02.006348 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:02.006350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:02.006437 (+ 87us) server_negotiation.cc:407] Connection header received
0805 22:43:02.006503 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:02.006504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:02.006517 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:02.006560 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:02.006945 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.007456 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.008227 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.008443 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.008500 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:02.011753 (+ 3253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:38357: BlockingRecv error: recv got EOF from 127.23.197.66:38357 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 05 22:43:02 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:43:02 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.67@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:43:02.141866 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:02.136285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.67:33287)
0805 22:43:02.136466 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:02.136478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:02.136548 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:02.136691 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:02.136692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:02.136936 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:02.136984 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:02.136986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.137678 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.137679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:02.138240 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:02.138241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.138611 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.138612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:02.138652 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:02.139230 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:02.139236 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:02.141795 (+ 2559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":127,"threads_started":1}
W20260805 22:43:02.142081 5377 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:02.136365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:33287 (local address 127.23.197.126:39127)
0805 22:43:02.136440 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:02.136443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:02.136550 (+ 107us) server_negotiation.cc:407] Connection header received
0805 22:43:02.136603 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:02.136604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:02.136618 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:02.136662 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:02.137056 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.137599 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.138340 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.138541 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.138600 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:02.141974 (+ 3374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.67:33287: BlockingRecv error: recv got EOF from 127.23.197.67:33287 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 05 22:43:02 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:43:02 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.65@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:43:02.898015 5383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:02.887503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.65:45809)
0805 22:43:02.887643 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:02.887658 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:02.887772 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:02.887980 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:02.887981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:02.888164 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:02.888216 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:02.888218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.889047 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.889048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:02.889578 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:02.889580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.890026 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.890027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:02.890079 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:02.895519 (+ 5440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:02.895527 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:02.897943 (+ 2416us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
W20260805 22:43:02.898165 5384 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:02.887610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:45809 (local address 127.23.197.126:39127)
0805 22:43:02.887844 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:02.887846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:02.887856 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:43:02.887880 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:02.887880 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:02.887892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:02.887960 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:02.888314 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.888968 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.889730 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:02.889939 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:02.890000 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:02.898079 (+ 8079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.65:45809: BlockingRecv error: recv got EOF from 127.23.197.65:45809 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":75,"threads_started":1}
Aug 05 22:43:03 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 22:43:03 dist-test-slave-g170 krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.197.66@KRBTEST.COM for oryx/127.23.197.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 22:43:03.017871 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:03.012265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:39127 (local address 127.23.197.66:58375)
0805 22:43:03.012498 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:03.012510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:03.012640 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:03.012926 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:03.012927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:03.013103 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:03.013155 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:03.013157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.013848 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.013849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:03.014592 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:03.014596 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.015230 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.015230 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:03.015280 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:03.015749 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:03.015754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:03.017734 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.197.126:39127: Server oryx/127.23.197.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":117,"threads_started":1}
W20260805 22:43:03.017912 5384 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:03.012391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:58375 (local address 127.23.197.126:39127)
0805 22:43:03.012483 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:03.012486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:03.012585 (+ 99us) server_negotiation.cc:407] Connection header received
0805 22:43:03.012720 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:03.012721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:03.012742 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:03.012890 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:03.013244 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.013755 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.014823 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.015122 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.015223 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:03.017844 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.197.66:58375: BlockingRecv error: recv got EOF from 127.23.197.66:58375 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 22:43:03.047816 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 4662
I20260805 22:43:03.062024 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 4881
I20260805 22:43:03.074875 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 5086
I20260805 22:43:03.086387 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 4556
2026-08-05T22:43:03Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17856 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-05T22:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:43:03Z Disabled control of system clock
I20260805 22:43:03.125851 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38387
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:38387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260805 22:43:03.215142 5393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:03.215387 5393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:03.215410 5393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:03.216681 5393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:43:03.216740 5393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:03.216755 5393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:03.216832 5393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:43:03.216862 5393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:43:03.218266 5393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:38387
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:03.218528 5393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:03.218830 5393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:03.224143 5405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:03.224108 5402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:03.224170 5403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:03.224639 5393 server_base.cc:1034] running on GCE node
I20260805 22:43:03.225023 5393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:03.225409 5393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:03.226585 5393 hybrid_clock.cc:648] HybridClock initialized: now 1785969783226574 us; error 32 us; skew 500 ppm
I20260805 22:43:03.228128 5393 webserver.cc:460] Webserver started at http://127.23.197.126:46301/ using document root <none> and password file <none>
I20260805 22:43:03.228407 5393 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:03.228488 5393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:03.228655 5393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:03.229920 5393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "99bd7b60943abf12390f76bccde4f220"
server_key_iv: "ae33843cd015e8d31408f106386c537a"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.230326 5393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "99bd7b60943abf12390f76bccde4f220"
server_key_iv: "ae33843cd015e8d31408f106386c537a"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.231781 5393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:43:03.232671 5416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.232909 5393 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:43:03.233024 5393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "99bd7b60943abf12390f76bccde4f220"
server_key_iv: "ae33843cd015e8d31408f106386c537a"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.233101 5393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:03.245302 5393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:03.245656 5393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:03.245831 5393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:03.250490 5393 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:38387
I20260805 22:43:03.250546 5503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:38387 every 8 connection(s)
I20260805 22:43:03.250949 5393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:43:03.251247 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 5393
I20260805 22:43:03.251389 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:43:03.251669 24341 external_mini_cluster.cc:1428] Setting key b397514abe10953813255c96e7ced80a
I20260805 22:43:03.251772 5505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:03.255769 5505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb: Bootstrap starting.
I20260805 22:43:03.256583 5505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:03.257061 5505 log.cc:824] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:03.257828 5505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb: No bootstrap required, opened a new log
I20260805 22:43:03.257838 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:03.252305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58670 (local address 127.23.197.126:38387)
0805 22:43:03.252536 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:03.252544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:03.252574 (+ 30us) server_negotiation.cc:407] Connection header received
0805 22:43:03.253092 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:03.253100 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:03.253174 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:03.253299 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:03.253564 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.254458 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.255622 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.256139 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.256241 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:03.257062 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:03.257066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:03.257066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:43:03.257283 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:03.257317 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:43:03.257327 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:03.257405 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:03.257488 (+ 83us) server_negotiation.cc:299] Negotiation successful
0805 22:43:03.257670 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
I20260805 22:43:03.259896 5505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38387 } }
I20260805 22:43:03.260062 5505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:03.260133 5505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b7f1d261b1e4fd4a69231dc1f82dceb, State: Initialized, Role: FOLLOWER
I20260805 22:43:03.260274 5505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38387 } }
I20260805 22:43:03.260401 5505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:43:03.260449 5505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:43:03.260499 5505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:03.261260 5505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38387 } }
I20260805 22:43:03.261410 5505 leader_election.cc:302] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b7f1d261b1e4fd4a69231dc1f82dceb; no voters:
I20260805 22:43:03.261607 5505 leader_election.cc:288] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:43:03.261662 5512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:43:03.261873 5512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [term 1 LEADER]: Becoming Leader. State: Replica: 6b7f1d261b1e4fd4a69231dc1f82dceb, State: Running, Role: LEADER
I20260805 22:43:03.261926 5505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:43:03.261976 5512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38387 } }
I20260805 22:43:03.262434 5513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38387 } } }
I20260805 22:43:03.262454 5515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b7f1d261b1e4fd4a69231dc1f82dceb. Latest consensus state: current_term: 1 leader_uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7f1d261b1e4fd4a69231dc1f82dceb" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38387 } } }
I20260805 22:43:03.262552 5513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [sys.catalog]: This master's current role is: LEADER
I20260805 22:43:03.262557 5515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb [sys.catalog]: This master's current role is: LEADER
I20260805 22:43:03.262868 5528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:43:03.263473 5528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:43:03.265177 5528 catalog_manager.cc:1347] Generated new cluster ID: b1d4bb873a74432086e0613047fe64dd
I20260805 22:43:03.265234 5528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:43:03.284004 5528 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:43:03.285197 5528 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:43:03.295267 5528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b7f1d261b1e4fd4a69231dc1f82dceb: Generated new TSK 0
I20260805 22:43:03.295509 5528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:43:03.300943 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38387
--builtin_ntp_servers=127.23.197.84:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260805 22:43:03.404407 5542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:03.404632 5542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:03.404655 5542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:03.405864 5542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:03.405910 5542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:03.405936 5542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:43:03.407320 5542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:03.407620 5542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:03.407948 5542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:03.413370 5555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:03.413409 5557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:03.413657 5542 server_base.cc:1034] running on GCE node
W20260805 22:43:03.413789 5560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:03.414021 5542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:03.414348 5542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:03.415547 5542 hybrid_clock.cc:648] HybridClock initialized: now 1785969783415514 us; error 50 us; skew 500 ppm
I20260805 22:43:03.417331 5542 webserver.cc:460] Webserver started at http://127.23.197.65:34703/ using document root <none> and password file <none>
I20260805 22:43:03.417636 5542 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:03.417701 5542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:03.417836 5542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:03.419363 5542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "1482b1063d9647c0934b6017ab8f8c5d"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "9d413f168be0b68b83dfc760a494f6d4"
server_key_iv: "620fecaa0084c20ab1e735baf3cef615"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.419786 5542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "1482b1063d9647c0934b6017ab8f8c5d"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "9d413f168be0b68b83dfc760a494f6d4"
server_key_iv: "620fecaa0084c20ab1e735baf3cef615"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.421639 5542 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260805 22:43:03.422619 5568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.422845 5542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:43:03.422941 5542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "1482b1063d9647c0934b6017ab8f8c5d"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "9d413f168be0b68b83dfc760a494f6d4"
server_key_iv: "620fecaa0084c20ab1e735baf3cef615"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.423020 5542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:03.441493 5542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:03.441823 5542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:03.441941 5542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:03.442174 5542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:03.442502 5542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:03.442545 5542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.442570 5542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:03.442584 5542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.449522 5542 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:39363
I20260805 22:43:03.449582 5740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:39363 every 8 connection(s)
I20260805 22:43:03.449962 5542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:43:03.456164 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:03.450921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:40031 (local address 127.23.197.126:38387)
0805 22:43:03.451003 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:03.451040 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:03.451308 (+ 268us) server_negotiation.cc:407] Connection header received
0805 22:43:03.451950 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:03.451951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:03.451971 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:03.452032 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:03.452843 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.453404 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.454537 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.454780 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.454877 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:03.455828 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:03.455829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:03.455830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:43:03.455835 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:03.455853 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:43:03.455860 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:03.455932 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:03.456034 (+ 102us) server_negotiation.cc:299] Negotiation successful
0805 22:43:03.456081 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 22:43:03.456377 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:03.450858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38387 (local address 127.23.197.65:40031)
0805 22:43:03.451149 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:03.451188 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:03.451815 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:03.452112 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:03.452115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:03.452128 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:03.452676 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:03.452681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.453515 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.453516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:03.454431 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:03.454433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.454890 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.454891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:03.455048 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:03.455656 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:43:03.455657 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:03.455687 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:43:03.455711 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:03.455933 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:03.455941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:03.455944 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:43:03.456036 (+ 92us) client_negotiation.cc:240] Negotiation successful
0805 22:43:03.456145 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":125,"threads_started":1}
I20260805 22:43:03.457129 5742 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38387
I20260805 22:43:03.457255 5742 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:03.457450 5742 heartbeater.cc:511] Master 127.23.197.126:38387 requested a full tablet report, sending...
I20260805 22:43:03.458021 5454 ts_manager.cc:194] Registered new tserver with Master: 1482b1063d9647c0934b6017ab8f8c5d (127.23.197.65:39363)
I20260805 22:43:03.458369 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 5542
I20260805 22:43:03.458483 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:43:03.458730 24341 external_mini_cluster.cc:1428] Setting key b76b153ca1ca9ca1a9f5ed4a8ebedcfe
I20260805 22:43:03.458706 5454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.65:40031
I20260805 22:43:03.459790 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38387
--builtin_ntp_servers=127.23.197.84:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260805 22:43:03.547821 5745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:03.548071 5745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:03.548108 5745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:03.549285 5745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:03.549358 5745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:03.549399 5745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:43:03.550797 5745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:03.551085 5745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:03.551425 5745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:03.556668 5758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:03.556669 5755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:03.556684 5756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:03.557001 5745 server_base.cc:1034] running on GCE node
I20260805 22:43:03.557220 5745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:03.557463 5745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:03.558630 5745 hybrid_clock.cc:648] HybridClock initialized: now 1785969783558613 us; error 28 us; skew 500 ppm
I20260805 22:43:03.560230 5745 webserver.cc:460] Webserver started at http://127.23.197.66:35693/ using document root <none> and password file <none>
I20260805 22:43:03.560540 5745 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:03.560606 5745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:03.560755 5745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:03.562189 5745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "827ad722ed9246a8a0874409f7c72e96"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "87aac2b29b318cbff5def667d9c66d4e"
server_key_iv: "cc32967e62c8637eb6bc11a3fdcc391a"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.562616 5745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "827ad722ed9246a8a0874409f7c72e96"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "87aac2b29b318cbff5def667d9c66d4e"
server_key_iv: "cc32967e62c8637eb6bc11a3fdcc391a"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.564260 5745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:43:03.565304 5766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.565512 5745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:43:03.565661 5745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "827ad722ed9246a8a0874409f7c72e96"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "87aac2b29b318cbff5def667d9c66d4e"
server_key_iv: "cc32967e62c8637eb6bc11a3fdcc391a"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.565745 5745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:03.586177 5745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:03.586570 5745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:03.586753 5745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:03.587033 5745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:03.587414 5745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:03.587473 5745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.587508 5745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:03.587538 5745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.594388 5745 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:41209
I20260805 22:43:03.594448 5931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:41209 every 8 connection(s)
I20260805 22:43:03.594879 5745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:43:03.596534 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 5745
I20260805 22:43:03.596652 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:43:03.597003 24341 external_mini_cluster.cc:1428] Setting key ad80e898b11ba695dff4dc4df3ec4764
I20260805 22:43:03.598218 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38387
--builtin_ntp_servers=127.23.197.84:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260805 22:43:03.602208 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:03.595776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:49907 (local address 127.23.197.126:38387)
0805 22:43:03.595863 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:03.595866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:03.596085 (+ 219us) server_negotiation.cc:407] Connection header received
0805 22:43:03.596962 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:03.596964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:03.596985 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:03.597047 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:03.598037 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.598682 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.600450 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.600790 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.600879 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:03.601866 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:03.601868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:03.601868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:43:03.601875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:03.601893 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:43:03.601897 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:03.601958 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:03.602051 (+ 93us) server_negotiation.cc:299] Negotiation successful
0805 22:43:03.602081 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 22:43:03.602362 5934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:03.595674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38387 (local address 127.23.197.66:49907)
0805 22:43:03.595968 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:03.596010 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:03.596770 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:03.597167 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:03.597172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:03.597185 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:03.597939 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:03.597947 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.598789 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.598790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:03.600330 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:03.600334 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.600894 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.600895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:03.601041 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:03.601712 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:43:03.601713 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:03.601744 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:43:03.601769 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:03.601965 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:03.601969 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:03.601972 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:43:03.602038 (+ 66us) client_negotiation.cc:240] Negotiation successful
0805 22:43:03.602123 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":113,"threads_started":1}
I20260805 22:43:03.602980 5932 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38387
I20260805 22:43:03.603092 5932 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:03.603343 5932 heartbeater.cc:511] Master 127.23.197.126:38387 requested a full tablet report, sending...
I20260805 22:43:03.604036 5454 ts_manager.cc:194] Registered new tserver with Master: 827ad722ed9246a8a0874409f7c72e96 (127.23.197.66:41209)
I20260805 22:43:03.604593 5454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.66:49907
W20260805 22:43:03.693105 5937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:03.693511 5937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:03.693619 5937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:03.695003 5937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:03.695084 5937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:03.695140 5937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:43:03.696720 5937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:03.697054 5937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:03.697413 5937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:03.703315 5944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:03.703393 5937 server_base.cc:1034] running on GCE node
W20260805 22:43:03.703405 5951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:03.703315 5945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:03.703838 5937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:03.704054 5937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:03.705232 5937 hybrid_clock.cc:648] HybridClock initialized: now 1785969783705209 us; error 50 us; skew 500 ppm
I20260805 22:43:03.706851 5937 webserver.cc:460] Webserver started at http://127.23.197.67:45019/ using document root <none> and password file <none>
I20260805 22:43:03.707154 5937 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:03.707206 5937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:03.707345 5937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:03.708480 5937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "c9465fb07bb14ab3b1cffb5647558bd7"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "056a8ebbc6bac402ba9b25f58a83f781"
server_key_iv: "52319dc82b0c0e291ac6db2148405e88"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.708899 5937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "c9465fb07bb14ab3b1cffb5647558bd7"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "056a8ebbc6bac402ba9b25f58a83f781"
server_key_iv: "52319dc82b0c0e291ac6db2148405e88"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.710355 5937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:43:03.711272 5959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.711517 5937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:43:03.711627 5937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "c9465fb07bb14ab3b1cffb5647558bd7"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "056a8ebbc6bac402ba9b25f58a83f781"
server_key_iv: "52319dc82b0c0e291ac6db2148405e88"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.711705 5937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:03.723456 5937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:03.723804 5937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:03.723968 5937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:03.724224 5937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:03.724587 5937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:03.724622 5937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.724645 5937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:03.724658 5937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.730996 5937 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:45841
I20260805 22:43:03.731084 6130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:45841 every 8 connection(s)
I20260805 22:43:03.731433 5937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:43:03.735698 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 5937
I20260805 22:43:03.735854 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:43:03.736088 24341 external_mini_cluster.cc:1428] Setting key 2f40a491ec90ee2890b10fdfa0a9ddab
I20260805 22:43:03.737334 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:03.732240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:40713 (local address 127.23.197.126:38387)
0805 22:43:03.732316 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:03.732318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:03.732579 (+ 261us) server_negotiation.cc:407] Connection header received
0805 22:43:03.733290 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:03.733291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:03.733304 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:03.733370 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:03.734094 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.734608 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.735615 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.735834 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.735922 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:03.737044 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:03.737046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:03.737046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:43:03.737050 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:03.737067 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:43:03.737070 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:03.737120 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:03.737252 (+ 132us) server_negotiation.cc:299] Negotiation successful
0805 22:43:03.737274 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 22:43:03.737586 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:03.732229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38387 (local address 127.23.197.67:40713)
0805 22:43:03.732438 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:03.732487 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:03.733166 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:03.733446 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:03.733449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:03.733457 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:03.733984 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:03.733988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.734711 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.734712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:03.735519 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:03.735521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:03.735963 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:03.735965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:03.736149 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:03.736889 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:43:03.736890 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:03.736920 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:43:03.736944 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:03.737126 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:03.737133 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:03.737136 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:43:03.737244 (+ 108us) client_negotiation.cc:240] Negotiation successful
0805 22:43:03.737349 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260805 22:43:03.738085 6132 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38387
I20260805 22:43:03.738180 6132 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:03.738341 6132 heartbeater.cc:511] Master 127.23.197.126:38387 requested a full tablet report, sending...
I20260805 22:43:03.738715 5454 ts_manager.cc:194] Registered new tserver with Master: c9465fb07bb14ab3b1cffb5647558bd7 (127.23.197.67:45841)
I20260805 22:43:03.739102 5454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.67:40713
I20260805 22:43:03.748533 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:43:03.750905 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 5542
W20260805 22:43:03.750912 5509 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:03.750139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58674 (local address 127.23.197.126:38387)
0805 22:43:03.750249 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:03.750251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:03.750343 (+ 92us) server_negotiation.cc:407] Connection header received
0805 22:43:03.750381 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:03.750382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:03.750392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:03.750443 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:03.750821 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58674: BlockingRecv error: recv got EOF from 127.0.0.1:58674 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 22:43:03.764148 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 5745
I20260805 22:43:03.772862 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 5937
I20260805 22:43:03.785295 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 5393
2026-08-05T22:43:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (697 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-05T22:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:43:03Z Disabled control of system clock
I20260805 22:43:03.821682 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:44095
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:44095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 22:43:03.906775 6144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:03.906983 6144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:03.907009 6144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:03.908087 6144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:43:03.908123 6144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:03.908134 6144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:03.908147 6144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:43:03.908159 6144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:43:03.909471 6144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:44095
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:44095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:03.909677 6144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:03.909941 6144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:03.915213 6156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:03.915308 6154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:03.915376 6144 server_base.cc:1034] running on GCE node
W20260805 22:43:03.915226 6153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:03.915958 6144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:03.916363 6144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:03.917546 6144 hybrid_clock.cc:648] HybridClock initialized: now 1785969783917513 us; error 54 us; skew 500 ppm
I20260805 22:43:03.919109 6144 webserver.cc:460] Webserver started at http://127.23.197.126:33191/ using document root <none> and password file <none>
I20260805 22:43:03.919407 6144 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:03.919482 6144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:03.919641 6144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:03.920854 6144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "baa0ce4c55d3f738a923bc2363c0184c"
server_key_iv: "4c4a6e2bf3a1a89683528858aa7a00a4"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.921263 6144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "baa0ce4c55d3f738a923bc2363c0184c"
server_key_iv: "4c4a6e2bf3a1a89683528858aa7a00a4"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.922631 6144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:43:03.923449 6164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:03.923653 6144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:43:03.923760 6144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae"
format_stamp: "Formatted at 2026-08-05 22:43:03 on dist-test-slave-g170"
server_key: "baa0ce4c55d3f738a923bc2363c0184c"
server_key_iv: "4c4a6e2bf3a1a89683528858aa7a00a4"
server_key_version: "encryptionkey@0"
I20260805 22:43:03.923847 6144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:03.927570 6144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:03.927852 6144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:03.927999 6144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:03.932200 6144 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:44095
I20260805 22:43:03.932230 6238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:44095 every 8 connection(s)
I20260805 22:43:03.932578 6144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:43:03.933336 6239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:03.936148 6239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae: Bootstrap starting.
I20260805 22:43:03.936551 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 6144
I20260805 22:43:03.936648 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:43:03.936822 6239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:03.936936 24341 external_mini_cluster.cc:1428] Setting key 908ae4667ff9dd128309960949ea3266
I20260805 22:43:03.937192 6239 log.cc:824] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:03.938002 6239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae: No bootstrap required, opened a new log
I20260805 22:43:03.939787 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:03.937520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.23.197.126:44095)
0805 22:43:03.937702 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:03.937712 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:03.937748 (+ 36us) server_negotiation.cc:407] Connection header received
0805 22:43:03.938348 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:03.938357 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:03.938408 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:03.938564 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:03.939282 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:03.939286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:03.939287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:43:03.939292 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:03.939338 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:43:03.939351 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:03.939448 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:03.939525 (+ 77us) server_negotiation.cc:299] Negotiation successful
0805 22:43:03.939582 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260805 22:43:03.940038 6239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44095 } }
I20260805 22:43:03.940199 6239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:03.940245 6239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57fbb9b43cfb487f8f139f0dacf9a5ae, State: Initialized, Role: FOLLOWER
I20260805 22:43:03.940371 6239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44095 } }
I20260805 22:43:03.940469 6239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:43:03.940510 6239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:43:03.940548 6239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:03.941474 6239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44095 } }
I20260805 22:43:03.941602 6239 leader_election.cc:302] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57fbb9b43cfb487f8f139f0dacf9a5ae; no voters:
I20260805 22:43:03.941776 6239 leader_election.cc:288] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:43:03.941859 6247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:43:03.942090 6239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:43:03.942102 6247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [term 1 LEADER]: Becoming Leader. State: Replica: 57fbb9b43cfb487f8f139f0dacf9a5ae, State: Running, Role: LEADER
I20260805 22:43:03.942231 6247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44095 } }
I20260805 22:43:03.942606 6248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44095 } } }
I20260805 22:43:03.942806 6249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57fbb9b43cfb487f8f139f0dacf9a5ae. Latest consensus state: current_term: 1 leader_uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57fbb9b43cfb487f8f139f0dacf9a5ae" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 44095 } } }
I20260805 22:43:03.942885 6248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [sys.catalog]: This master's current role is: LEADER
I20260805 22:43:03.942901 6249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae [sys.catalog]: This master's current role is: LEADER
I20260805 22:43:03.943198 6259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:43:03.943701 6259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:43:03.945425 6259 catalog_manager.cc:1347] Generated new cluster ID: 722f9d110ba1431c8f1016c3eb979437
I20260805 22:43:03.945477 6259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:43:03.964322 6259 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:43:03.964547 6259 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:43:03.971451 6259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57fbb9b43cfb487f8f139f0dacf9a5ae: Generated new TSK 0
I20260805 22:43:03.971787 6259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:43:03.973187 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44095
--builtin_ntp_servers=127.23.197.84:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 22:43:04.063211 6274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:04.063463 6274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:04.063500 6274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:04.064703 6274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:04.064767 6274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:04.064837 6274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:43:04.066226 6274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:04.066495 6274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:04.066815 6274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:04.072136 6280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:04.072193 6284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:04.072309 6281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:04.072440 6274 server_base.cc:1034] running on GCE node
I20260805 22:43:04.072682 6274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:04.072983 6274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:04.074159 6274 hybrid_clock.cc:648] HybridClock initialized: now 1785969784074143 us; error 32 us; skew 500 ppm
I20260805 22:43:04.079906 6274 webserver.cc:460] Webserver started at http://127.23.197.65:33943/ using document root <none> and password file <none>
I20260805 22:43:04.080204 6274 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:04.080284 6274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:04.080448 6274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:04.081652 6274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "8a58bcb9c7294cebbf6181009487d784"
format_stamp: "Formatted at 2026-08-05 22:43:04 on dist-test-slave-g170"
server_key: "9a00ef534739bb655e1c7a57a578eeb0"
server_key_iv: "321c72656f74541f342238b02d026bdc"
server_key_version: "encryptionkey@0"
I20260805 22:43:04.082027 6274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "8a58bcb9c7294cebbf6181009487d784"
format_stamp: "Formatted at 2026-08-05 22:43:04 on dist-test-slave-g170"
server_key: "9a00ef534739bb655e1c7a57a578eeb0"
server_key_iv: "321c72656f74541f342238b02d026bdc"
server_key_version: "encryptionkey@0"
I20260805 22:43:04.083451 6274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:43:04.084316 6295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:04.084574 6274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:43:04.084672 6274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "8a58bcb9c7294cebbf6181009487d784"
format_stamp: "Formatted at 2026-08-05 22:43:04 on dist-test-slave-g170"
server_key: "9a00ef534739bb655e1c7a57a578eeb0"
server_key_iv: "321c72656f74541f342238b02d026bdc"
server_key_version: "encryptionkey@0"
I20260805 22:43:04.084731 6274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:04.088164 6274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:04.088389 6274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:04.088490 6274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:04.088650 6274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:04.089003 6274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:04.089038 6274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:04.089059 6274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:04.089072 6274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:04.094792 6274 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:39483
I20260805 22:43:04.094871 6459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:39483 every 8 connection(s)
I20260805 22:43:04.095211 6274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:43:04.098497 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:04.096020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:51747 (local address 127.23.197.126:44095)
0805 22:43:04.096089 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:04.096091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:04.096335 (+ 244us) server_negotiation.cc:407] Connection header received
0805 22:43:04.096899 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:04.096901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:04.096908 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:04.096960 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:04.098193 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:04.098194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:04.098195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:43:04.098199 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:04.098216 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:43:04.098220 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:04.098297 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:04.098406 (+ 109us) server_negotiation.cc:299] Negotiation successful
0805 22:43:04.098423 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 22:43:04.098706 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:04.096016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:44095 (local address 127.23.197.65:51747)
0805 22:43:04.096212 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:04.096246 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:04.096748 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:04.097034 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:04.097038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:04.097047 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:04.098030 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:43:04.098030 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:04.098059 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:43:04.098088 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:04.098297 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:04.098304 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:04.098307 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:43:04.098417 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 22:43:04.098481 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260805 22:43:04.099161 6461 heartbeater.cc:347] Connected to a master server at 127.23.197.126:44095
I20260805 22:43:04.099252 6461 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:04.099416 6461 heartbeater.cc:511] Master 127.23.197.126:44095 requested a full tablet report, sending...
I20260805 22:43:04.099522 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 6274
I20260805 22:43:04.099627 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:43:04.099855 24341 external_mini_cluster.cc:1428] Setting key b02ac5796d13914f7436507d8f52c49a
I20260805 22:43:04.100078 6189 ts_manager.cc:194] Registered new tserver with Master: 8a58bcb9c7294cebbf6181009487d784 (127.23.197.65:39483)
I20260805 22:43:04.101092 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44095
--builtin_ntp_servers=127.23.197.84:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 22:43:04.185721 6465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:04.185981 6465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:04.186020 6465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:04.187176 6465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:04.187286 6465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:04.187336 6465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:43:04.188802 6465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:04.189064 6465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:04.189386 6465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:04.194823 6476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:04.194875 6473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:04.194897 6474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:04.195166 6465 server_base.cc:1034] running on GCE node
I20260805 22:43:04.195411 6465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:04.195672 6465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:04.196842 6465 hybrid_clock.cc:648] HybridClock initialized: now 1785969784196814 us; error 42 us; skew 500 ppm
I20260805 22:43:04.198187 6465 webserver.cc:460] Webserver started at http://127.23.197.66:44413/ using document root <none> and password file <none>
I20260805 22:43:04.198446 6465 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:04.198493 6465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:04.198673 6465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:04.199777 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "6f53c309a8c043d3a4edf7c2a48e607a"
format_stamp: "Formatted at 2026-08-05 22:43:04 on dist-test-slave-g170"
server_key: "c8f97662512924c02217dfc8c98198e7"
server_key_iv: "9f61f838395ef3793caf4355f2cf4054"
server_key_version: "encryptionkey@0"
I20260805 22:43:04.200155 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "6f53c309a8c043d3a4edf7c2a48e607a"
format_stamp: "Formatted at 2026-08-05 22:43:04 on dist-test-slave-g170"
server_key: "c8f97662512924c02217dfc8c98198e7"
server_key_iv: "9f61f838395ef3793caf4355f2cf4054"
server_key_version: "encryptionkey@0"
I20260805 22:43:04.201649 6465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:43:04.202502 6484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:04.202704 6465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:43:04.202812 6465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "6f53c309a8c043d3a4edf7c2a48e607a"
format_stamp: "Formatted at 2026-08-05 22:43:04 on dist-test-slave-g170"
server_key: "c8f97662512924c02217dfc8c98198e7"
server_key_iv: "9f61f838395ef3793caf4355f2cf4054"
server_key_version: "encryptionkey@0"
I20260805 22:43:04.202889 6465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:04.206409 6465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:04.206683 6465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:04.206825 6465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:04.207050 6465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:04.207378 6465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:04.207432 6465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:04.207472 6465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:04.207499 6465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:04.213696 6465 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:44937
I20260805 22:43:04.213744 6648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:44937 every 8 connection(s)
I20260805 22:43:04.214089 6465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:43:04.216959 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 6465
I20260805 22:43:04.217090 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:43:04.217308 24341 external_mini_cluster.cc:1428] Setting key e2d35c487b030eea083df5e2e3abb2cd
I20260805 22:43:04.217710 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:04.214854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:35481 (local address 127.23.197.126:44095)
0805 22:43:04.214921 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:04.214924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:04.215169 (+ 245us) server_negotiation.cc:407] Connection header received
0805 22:43:04.215760 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:04.215762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:04.215769 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:04.215844 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:04.217288 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:04.217292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:04.217292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:43:04.217302 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:04.217319 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:43:04.217324 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:04.217411 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:04.217541 (+ 130us) server_negotiation.cc:299] Negotiation successful
0805 22:43:04.217559 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 22:43:04.217840 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:04.214832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:44095 (local address 127.23.197.66:35481)
0805 22:43:04.215044 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:04.215079 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:04.215607 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:04.215934 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:04.215938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:04.215946 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:04.217028 (+ 1082us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:43:04.217029 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:04.217078 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:43:04.217117 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:04.217424 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:04.217432 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:04.217434 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:43:04.217530 (+ 96us) client_negotiation.cc:240] Negotiation successful
0805 22:43:04.217585 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260805 22:43:04.218353 6649 heartbeater.cc:347] Connected to a master server at 127.23.197.126:44095
I20260805 22:43:04.218459 6649 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:04.218618 6649 heartbeater.cc:511] Master 127.23.197.126:44095 requested a full tablet report, sending...
I20260805 22:43:04.218617 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44095
--builtin_ntp_servers=127.23.197.84:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260805 22:43:04.219053 6188 ts_manager.cc:194] Registered new tserver with Master: 6f53c309a8c043d3a4edf7c2a48e607a (127.23.197.66:44937)
W20260805 22:43:04.328132 6652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:04.328469 6652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:04.328518 6652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:04.330154 6652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:04.330206 6652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:04.330236 6652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:43:04.331604 6652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:44095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:04.331815 6652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:04.332080 6652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:04.337419 6667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:04.337448 6658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:04.337421 6663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:04.337589 6652 server_base.cc:1034] running on GCE node
I20260805 22:43:04.337896 6652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:04.338157 6652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:04.339359 6652 hybrid_clock.cc:648] HybridClock initialized: now 1785969784339357 us; error 41 us; skew 500 ppm
I20260805 22:43:04.340883 6652 webserver.cc:460] Webserver started at http://127.23.197.67:44063/ using document root <none> and password file <none>
I20260805 22:43:04.341161 6652 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:04.341238 6652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:04.341414 6652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:04.342818 6652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "62cfe2f9dc0b45cea40c0f98e692df4e"
format_stamp: "Formatted at 2026-08-05 22:43:04 on dist-test-slave-g170"
server_key: "89f98cbbd17c8c4524544e81f5e67838"
server_key_iv: "1ec6e8c079e292a512b9402f46321ec2"
server_key_version: "encryptionkey@0"
I20260805 22:43:04.343186 6652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "62cfe2f9dc0b45cea40c0f98e692df4e"
format_stamp: "Formatted at 2026-08-05 22:43:04 on dist-test-slave-g170"
server_key: "89f98cbbd17c8c4524544e81f5e67838"
server_key_iv: "1ec6e8c079e292a512b9402f46321ec2"
server_key_version: "encryptionkey@0"
I20260805 22:43:04.344655 6652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260805 22:43:04.345868 6677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:04.346158 6652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:43:04.346266 6652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "62cfe2f9dc0b45cea40c0f98e692df4e"
format_stamp: "Formatted at 2026-08-05 22:43:04 on dist-test-slave-g170"
server_key: "89f98cbbd17c8c4524544e81f5e67838"
server_key_iv: "1ec6e8c079e292a512b9402f46321ec2"
server_key_version: "encryptionkey@0"
I20260805 22:43:04.346376 6652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:04.350019 6652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:04.350319 6652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:04.350471 6652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:04.350688 6652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:04.351027 6652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:04.351085 6652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:04.351125 6652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:04.351153 6652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:04.357764 6652 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:45557
I20260805 22:43:04.357892 6855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:45557 every 8 connection(s)
I20260805 22:43:04.358248 6652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:43:04.361685 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:04.359132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:52623 (local address 127.23.197.126:44095)
0805 22:43:04.359208 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:04.359210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:04.359480 (+ 270us) server_negotiation.cc:407] Connection header received
0805 22:43:04.360068 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:04.360069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:04.360076 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:04.360126 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:04.361409 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:04.361410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:04.361410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:43:04.361416 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:04.361431 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:43:04.361436 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:04.361496 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:04.361591 (+ 95us) server_negotiation.cc:299] Negotiation successful
0805 22:43:04.361608 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 22:43:04.361927 6859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:04.359116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:44095 (local address 127.23.197.67:52623)
0805 22:43:04.359351 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:04.359386 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:04.359953 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:04.360206 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:04.360210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:04.360218 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:04.361248 (+ 1030us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:43:04.361248 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:04.361278 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:43:04.361307 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:04.361497 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:04.361503 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:04.361506 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:43:04.361604 (+ 98us) client_negotiation.cc:240] Negotiation successful
0805 22:43:04.361669 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":112,"threads_started":1}
I20260805 22:43:04.362581 6856 heartbeater.cc:347] Connected to a master server at 127.23.197.126:44095
I20260805 22:43:04.362695 6856 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:04.362969 6856 heartbeater.cc:511] Master 127.23.197.126:44095 requested a full tablet report, sending...
I20260805 22:43:04.363484 6188 ts_manager.cc:194] Registered new tserver with Master: 62cfe2f9dc0b45cea40c0f98e692df4e (127.23.197.67:45557)
I20260805 22:43:04.366568 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 6652
I20260805 22:43:04.366701 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:43:04.366993 24341 external_mini_cluster.cc:1428] Setting key a3d3a691fb56a66f0e7e64abdfcc5212
I20260805 22:43:04.368062 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260805 22:43:04.370124 6245 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:43:04.369538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.23.197.126:44095)
0805 22:43:04.369642 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:04.369645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:04.369759 (+ 114us) server_negotiation.cc:407] Connection header received
0805 22:43:04.369837 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:04.369838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:04.369843 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:04.369903 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:04.370054 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36090: BlockingRecv error: recv got EOF from 127.0.0.1:36090 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260805 22:43:04.370186 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 6274
I20260805 22:43:04.396035 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 6465
I20260805 22:43:04.411404 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 6652
I20260805 22:43:04.427845 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 6144
2026-08-05T22:43:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (647 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:43:04 dist-test-slave-g170 krb5kdc[6870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:43:04 dist-test-slave-g170 krb5kdc[6870](info): set up 2 sockets
Aug 05 22:43:04 dist-test-slave-g170 krb5kdc[6870](info): commencing operation
krb5kdc: starting...
W20260805 22:43:05.687611 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:43:05 dist-test-slave-g170 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:43:05.823755 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40907
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:40907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:43:05.911252 6889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:05.911521 6889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:05.911562 6889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:05.912783 6889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:43:05.912858 6889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:05.912881 6889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:05.912904 6889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:43:05.912933 6889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:43:05.914490 6889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:40907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:05.914795 6889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:05.915040 6889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:05.920447 6894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:05.920485 6901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:05.920493 6889 server_base.cc:1034] running on GCE node
W20260805 22:43:05.920514 6895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:05.920962 6889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:05.921324 6889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:05.922523 6889 hybrid_clock.cc:648] HybridClock initialized: now 1785969785922485 us; error 61 us; skew 500 ppm
Aug 05 22:43:05 dist-test-slave-g170 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:05.925490 6889 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:43:05.925964 6889 webserver.cc:460] Webserver started at http://127.23.197.126:42303/ using document root <none> and password file <none>
I20260805 22:43:05.926177 6889 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:05.926226 6889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:05.926328 6889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:05.927508 6889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "90c9632385cf4dcbba98f2dc276e0b35"
format_stamp: "Formatted at 2026-08-05 22:43:05 on dist-test-slave-g170"
server_key: "2b881e75924b97e70b40a1c1e4068869"
server_key_iv: "e78a2e2384c5346931d5cc8ad3ae5782"
server_key_version: "encryptionkey@0"
I20260805 22:43:05.927873 6889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "90c9632385cf4dcbba98f2dc276e0b35"
format_stamp: "Formatted at 2026-08-05 22:43:05 on dist-test-slave-g170"
server_key: "2b881e75924b97e70b40a1c1e4068869"
server_key_iv: "e78a2e2384c5346931d5cc8ad3ae5782"
server_key_version: "encryptionkey@0"
I20260805 22:43:05.929592 6889 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260805 22:43:05.930495 6908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:05.930790 6889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:43:05.930934 6889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "90c9632385cf4dcbba98f2dc276e0b35"
format_stamp: "Formatted at 2026-08-05 22:43:05 on dist-test-slave-g170"
server_key: "2b881e75924b97e70b40a1c1e4068869"
server_key_iv: "e78a2e2384c5346931d5cc8ad3ae5782"
server_key_version: "encryptionkey@0"
I20260805 22:43:05.931023 6889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:05.944315 6889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:05.946816 6889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:05.947014 6889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:05.951788 6889 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:40907
I20260805 22:43:05.951861 6985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:40907 every 8 connection(s)
I20260805 22:43:05.952273 6889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:43:05.953557 6987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:05.956156 6987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35: Bootstrap starting.
I20260805 22:43:05.956872 6987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:05.957221 6987 log.cc:824] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:05.958103 6987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35: No bootstrap required, opened a new log
I20260805 22:43:05.959820 6987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9632385cf4dcbba98f2dc276e0b35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40907 } }
I20260805 22:43:05.959967 6987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:05.959990 6987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c9632385cf4dcbba98f2dc276e0b35, State: Initialized, Role: FOLLOWER
I20260805 22:43:05.960153 6987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9632385cf4dcbba98f2dc276e0b35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40907 } }
I20260805 22:43:05.960239 6987 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:43:05.960280 6987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:43:05.960312 6987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:05.960522 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 6889
I20260805 22:43:05.960649 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:43:05.960918 24341 external_mini_cluster.cc:1428] Setting key 01a2345fb861bdcd216a8bebce2ca243
I20260805 22:43:05.961552 6987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9632385cf4dcbba98f2dc276e0b35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40907 } }
I20260805 22:43:05.961699 6987 leader_election.cc:302] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90c9632385cf4dcbba98f2dc276e0b35; no voters:
I20260805 22:43:05.961889 6987 leader_election.cc:288] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:43:05.961951 6993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:43:05.962116 6993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [term 1 LEADER]: Becoming Leader. State: Replica: 90c9632385cf4dcbba98f2dc276e0b35, State: Running, Role: LEADER
I20260805 22:43:05.962160 6987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:43:05.962227 6993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9632385cf4dcbba98f2dc276e0b35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40907 } }
I20260805 22:43:05.962584 6995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90c9632385cf4dcbba98f2dc276e0b35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9632385cf4dcbba98f2dc276e0b35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40907 } } }
I20260805 22:43:05.962698 6995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [sys.catalog]: This master's current role is: LEADER
I20260805 22:43:05.962574 6996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90c9632385cf4dcbba98f2dc276e0b35. Latest consensus state: current_term: 1 leader_uuid: "90c9632385cf4dcbba98f2dc276e0b35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c9632385cf4dcbba98f2dc276e0b35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40907 } } }
I20260805 22:43:05.962875 6996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35 [sys.catalog]: This master's current role is: LEADER
W20260805 22:43:05.964135 7016 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:43:05.964197 7016 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:43:05.964550 7004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:43:05.964756 7004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:43:05.966657 7004 catalog_manager.cc:1347] Generated new cluster ID: ce82eb28b7ba495892eedade18e7c6d7
I20260805 22:43:05.966714 7004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:43:05 dist-test-slave-g170 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:43:05.974088 6991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:05.961524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.23.197.126:40907)
0805 22:43:05.961795 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:05.961807 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:05.961849 (+ 42us) server_negotiation.cc:407] Connection header received
0805 22:43:05.961982 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:05.961991 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:05.962068 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:05.963034 (+ 966us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:05.963128 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:05.964743 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:05.965783 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:05.966406 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:05.966527 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:05.969623 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:05.969628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:05.969629 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:05.969644 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:05.972098 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:05.972529 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:05.972533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:05.972534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:05.972711 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:05.972970 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:05.972971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:05.972971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:05.973376 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:05.973574 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:05.973655 (+ 81us) server_negotiation.cc:299] Negotiation successful
0805 22:43:05.973835 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":111,"threads_started":1}
I20260805 22:43:05.979065 7004 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:43:05.979871 7004 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:43:05.983199 7004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35: Generated new TSK 0
I20260805 22:43:05.983417 7004 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:43:06.016743 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40907
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:43:06.104714 7025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:06.105007 7025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:06.105034 7025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:06.106257 7025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:06.106299 7025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:06.106328 7025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:43:06.107837 7025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:06.108105 7025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:06.108366 7025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:06.113523 7034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:06.113528 7033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:06.113698 7025 server_base.cc:1034] running on GCE node
W20260805 22:43:06.113523 7036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:06.114080 7025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:06.114359 7025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:06.115562 7025 hybrid_clock.cc:648] HybridClock initialized: now 1785969786115560 us; error 43 us; skew 500 ppm
Aug 05 22:43:06 dist-test-slave-g170 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:06.118628 7025 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:43:06.119168 7025 webserver.cc:460] Webserver started at http://127.23.197.65:38801/ using document root <none> and password file <none>
I20260805 22:43:06.119477 7025 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:06.119556 7025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:06.119736 7025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:06.120981 7025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "f664494af7b246c7b4a9133136ac1cca"
format_stamp: "Formatted at 2026-08-05 22:43:06 on dist-test-slave-g170"
server_key: "95f6f6c9f451c4216615e5a31727b55f"
server_key_iv: "751ba7977eda52cdf3149c2e8f28b536"
server_key_version: "encryptionkey@0"
I20260805 22:43:06.121417 7025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "f664494af7b246c7b4a9133136ac1cca"
format_stamp: "Formatted at 2026-08-05 22:43:06 on dist-test-slave-g170"
server_key: "95f6f6c9f451c4216615e5a31727b55f"
server_key_iv: "751ba7977eda52cdf3149c2e8f28b536"
server_key_version: "encryptionkey@0"
I20260805 22:43:06.123018 7025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:43:06.123850 7049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:06.124069 7025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:43:06.124204 7025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "f664494af7b246c7b4a9133136ac1cca"
format_stamp: "Formatted at 2026-08-05 22:43:06 on dist-test-slave-g170"
server_key: "95f6f6c9f451c4216615e5a31727b55f"
server_key_iv: "751ba7977eda52cdf3149c2e8f28b536"
server_key_version: "encryptionkey@0"
I20260805 22:43:06.124290 7025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:06.139415 7025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:06.141901 7025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:06.142104 7025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:06.142390 7025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:06.142771 7025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:06.142833 7025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:06.142881 7025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:06.142930 7025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:06.150012 7025 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:45357
I20260805 22:43:06.150054 7213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:45357 every 8 connection(s)
I20260805 22:43:06.150451 7025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:43:06.153366 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 7025
I20260805 22:43:06.153487 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:43:06.153666 24341 external_mini_cluster.cc:1428] Setting key bfdcdce3de7bee0b4c3fcf893d0d9f75
Aug 05 22:43:06 dist-test-slave-g170 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:43:06.161542 6991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.151337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:46179 (local address 127.23.197.126:40907)
0805 22:43:06.151423 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.151425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.151659 (+ 234us) server_negotiation.cc:407] Connection header received
0805 22:43:06.152204 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.152205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.152218 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.152287 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:06.153581 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.154225 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.155326 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.155558 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.155657 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.158118 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:06.158120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:06.158120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:06.158129 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:06.160012 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:06.160466 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:06.160468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:06.160468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:06.160535 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:06.160819 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:06.160821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:06.160821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:06.160945 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:06.161020 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.161343 (+ 323us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.161446 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 22:43:06.161739 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.151322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40907 (local address 127.23.197.65:46179)
0805 22:43:06.151524 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:06.151563 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:06.152098 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:06.152371 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:06.152375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:06.153061 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:06.153372 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.153380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.154367 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.154368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.155206 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.155209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.155673 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.155674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.155823 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.156446 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:06.156455 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:06.158004 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:06.160122 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:06.160132 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:06.160134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:06.160382 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:06.160607 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:06.160608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:06.160608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:06.160728 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:06.161023 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:06.161031 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:06.161196 (+ 165us) client_negotiation.cc:769] Sending connection context
0805 22:43:06.161305 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 22:43:06.161478 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":113,"threads_started":1}
I20260805 22:43:06.162741 7214 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40907
I20260805 22:43:06.162894 7214 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:06.163113 7214 heartbeater.cc:511] Master 127.23.197.126:40907 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
I20260805 22:43:06.163740 6930 ts_manager.cc:194] Registered new tserver with Master: f664494af7b246c7b4a9133136ac1cca (127.23.197.65:45357)
I20260805 22:43:06.164427 6930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:46179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:43:06.184976 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40907
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:43:06.271466 7221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:06.271869 7221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:06.271934 7221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:06.273979 7221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:06.274050 7221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:06.274092 7221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:43:06.276098 7221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:06.276321 7221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:06.276535 7221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:06.282020 7227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:06.281977 7231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:06.281975 7228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:06.282195 7221 server_base.cc:1034] running on GCE node
I20260805 22:43:06.282548 7221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:06.282847 7221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:06.284035 7221 hybrid_clock.cc:648] HybridClock initialized: now 1785969786284034 us; error 46 us; skew 500 ppm
Aug 05 22:43:06 dist-test-slave-g170 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:06.286921 7221 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:43:06.287412 7221 webserver.cc:460] Webserver started at http://127.23.197.66:39445/ using document root <none> and password file <none>
I20260805 22:43:06.287662 7221 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:06.287744 7221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:06.287909 7221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:06.289112 7221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "0b9b5420e9cb42de8eaf55f9d70339fb"
format_stamp: "Formatted at 2026-08-05 22:43:06 on dist-test-slave-g170"
server_key: "7d403f60057653ac89bcd413bf4e80a4"
server_key_iv: "ee4e9b659e181ea944c0d0a947032ca7"
server_key_version: "encryptionkey@0"
I20260805 22:43:06.289497 7221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "0b9b5420e9cb42de8eaf55f9d70339fb"
format_stamp: "Formatted at 2026-08-05 22:43:06 on dist-test-slave-g170"
server_key: "7d403f60057653ac89bcd413bf4e80a4"
server_key_iv: "ee4e9b659e181ea944c0d0a947032ca7"
server_key_version: "encryptionkey@0"
I20260805 22:43:06.290984 7221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260805 22:43:06.291787 7243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:06.291999 7221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:43:06.292110 7221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "0b9b5420e9cb42de8eaf55f9d70339fb"
format_stamp: "Formatted at 2026-08-05 22:43:06 on dist-test-slave-g170"
server_key: "7d403f60057653ac89bcd413bf4e80a4"
server_key_iv: "ee4e9b659e181ea944c0d0a947032ca7"
server_key_version: "encryptionkey@0"
I20260805 22:43:06.292186 7221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:06.308874 7221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:06.310997 7221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:06.311157 7221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:06.311414 7221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:06.311777 7221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:06.311844 7221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:06.311889 7221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:06.311923 7221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:06.318147 7221 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:34493
I20260805 22:43:06.318310 7401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:34493 every 8 connection(s)
I20260805 22:43:06.318593 7221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:43:06.321225 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 7221
I20260805 22:43:06.321339 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:43:06.321574 24341 external_mini_cluster.cc:1428] Setting key 576a154a2f5c7986a396fe399564aa8e
Aug 05 22:43:06 dist-test-slave-g170 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:43:06.329293 6991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.319327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:33635 (local address 127.23.197.126:40907)
0805 22:43:06.319393 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.319413 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.319627 (+ 214us) server_negotiation.cc:407] Connection header received
0805 22:43:06.320183 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.320185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.320199 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.320252 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:06.321476 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.322036 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.323112 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.323445 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.323527 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.326030 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:06.326032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:06.326032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:06.326043 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:06.327801 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:06.328218 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:06.328219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:06.328219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:06.328289 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:06.328566 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:06.328567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:06.328567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:06.328710 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:06.328805 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.329096 (+ 291us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.329199 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 22:43:06.329520 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.319310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40907 (local address 127.23.197.66:33635)
0805 22:43:06.319499 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:06.319535 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:06.320063 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:06.320322 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:06.320326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:06.320988 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:06.321306 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.321314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.322133 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.322135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.322997 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.323000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.323536 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.323537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.323637 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.324239 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:06.324251 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:06.325924 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:06.327887 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:06.327894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:06.327897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:06.328133 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:06.328365 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:06.328366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:06.328366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:06.328498 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:06.328794 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:06.328801 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:06.328954 (+ 153us) client_negotiation.cc:769] Sending connection context
0805 22:43:06.329074 (+ 120us) client_negotiation.cc:240] Negotiation successful
0805 22:43:06.329242 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260805 22:43:06.330073 7402 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40907
I20260805 22:43:06.330197 7402 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:06.330399 7402 heartbeater.cc:511] Master 127.23.197.126:40907 requested a full tablet report, sending...
I20260805 22:43:06.330833 6930 ts_manager.cc:194] Registered new tserver with Master: 0b9b5420e9cb42de8eaf55f9d70339fb (127.23.197.66:34493)
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
I20260805 22:43:06.331259 6930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:33635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:43:06.351799 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40907
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:43:06.438405 7410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:06.438647 7410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:06.438691 7410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:06.439904 7410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:06.439973 7410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:06.440021 7410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:43:06.441686 7410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:06.441963 7410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:06.442203 7410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:06.447542 7419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:06.447551 7422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:06.447551 7418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:06.447742 7410 server_base.cc:1034] running on GCE node
I20260805 22:43:06.448000 7410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:06.448297 7410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:06.449484 7410 hybrid_clock.cc:648] HybridClock initialized: now 1785969786449469 us; error 30 us; skew 500 ppm
Aug 05 22:43:06 dist-test-slave-g170 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:06.452315 7410 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:43:06.452934 7410 webserver.cc:460] Webserver started at http://127.23.197.67:33781/ using document root <none> and password file <none>
I20260805 22:43:06.453198 7410 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:06.453284 7410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:06.453464 7410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:43:06.454671 7410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "ed69eb945b6245d7a56247b0fd493b98"
format_stamp: "Formatted at 2026-08-05 22:43:06 on dist-test-slave-g170"
server_key: "192f7306325326627ad710411e598e97"
server_key_iv: "d9faf3edd8ead2372abb951a08f0deff"
server_key_version: "encryptionkey@0"
I20260805 22:43:06.455094 7410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "ed69eb945b6245d7a56247b0fd493b98"
format_stamp: "Formatted at 2026-08-05 22:43:06 on dist-test-slave-g170"
server_key: "192f7306325326627ad710411e598e97"
server_key_iv: "d9faf3edd8ead2372abb951a08f0deff"
server_key_version: "encryptionkey@0"
I20260805 22:43:06.456662 7410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:43:06.457653 7432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:06.457937 7410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:43:06.458051 7410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "ed69eb945b6245d7a56247b0fd493b98"
format_stamp: "Formatted at 2026-08-05 22:43:06 on dist-test-slave-g170"
server_key: "192f7306325326627ad710411e598e97"
server_key_iv: "d9faf3edd8ead2372abb951a08f0deff"
server_key_version: "encryptionkey@0"
I20260805 22:43:06.458135 7410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:06.475420 7410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:06.478039 7410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:06.478233 7410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:06.478513 7410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:06.478857 7410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:06.478914 7410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:06.478961 7410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:06.478993 7410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:06.485838 7410 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:38905
I20260805 22:43:06.485925 7596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:38905 every 8 connection(s)
I20260805 22:43:06.486290 7410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:43:06.488021 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 7410
I20260805 22:43:06.488137 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:43:06.488462 24341 external_mini_cluster.cc:1428] Setting key 3305592c18790c4850fd3a6b3473a4bd
Aug 05 22:43:06 dist-test-slave-g170 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:43:06.496682 6991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.487113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:44733 (local address 127.23.197.126:40907)
0805 22:43:06.487184 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.487187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.487435 (+ 248us) server_negotiation.cc:407] Connection header received
0805 22:43:06.488072 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.488073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.488105 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.488171 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:06.489630 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.490133 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.491144 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.491351 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.491412 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.493843 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:06.493845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:06.493845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:06.493853 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:06.495275 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:06.495672 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:06.495673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:06.495673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:06.495731 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:06.495964 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:06.495964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:06.495964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:06.496067 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:06.496145 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.496528 (+ 383us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.496603 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 22:43:06.496940 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.487095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40907 (local address 127.23.197.67:44733)
0805 22:43:06.487300 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:06.487341 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:06.487961 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:06.488264 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:06.488269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:06.489203 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:06.489509 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.489515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.490230 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.490230 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.491045 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.491047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.491443 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.491443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.491561 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.492196 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:06.492211 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:06.493751 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:06.495349 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:06.495356 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:06.495358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:06.495599 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:06.495796 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:06.495796 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:06.495797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:06.495907 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:06.496214 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:06.496226 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:06.496358 (+ 132us) client_negotiation.cc:769] Sending connection context
0805 22:43:06.496479 (+ 121us) client_negotiation.cc:240] Negotiation successful
0805 22:43:06.496635 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":127,"threads_started":1}
I20260805 22:43:06.497712 7597 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40907
I20260805 22:43:06.497820 7597 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:06.498034 7597 heartbeater.cc:511] Master 127.23.197.126:40907 requested a full tablet report, sending...
I20260805 22:43:06.498428 6930 ts_manager.cc:194] Registered new tserver with Master: ed69eb945b6245d7a56247b0fd493b98 (127.23.197.67:38905)
I20260805 22:43:06.498854 6930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:44733
I20260805 22:43:06.500154 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:43:06.508116 6991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.501666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.23.197.126:40907)
0805 22:43:06.501743 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.501745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.501914 (+ 169us) server_negotiation.cc:407] Connection header received
0805 22:43:06.501973 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.501973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.501983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.502029 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:06.502639 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.503174 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.503909 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.504123 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.504181 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.505565 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:06.505566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:06.505567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:06.505573 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:06.506904 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:06.507264 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:06.507265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:06.507265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:06.507322 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:06.507553 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:06.507554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:06.507554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:06.507720 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:06.507806 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.507925 (+ 119us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.507990 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 22:43:06.509977 6930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:43:06.511518 6930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:43:06.518031 7620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.514420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.23.197.66:34493)
0805 22:43:06.514630 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.514635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.514649 (+ 14us) server_negotiation.cc:407] Connection header received
0805 22:43:06.514725 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.514728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.514747 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.514839 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:06.515042 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.515767 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.517090 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.517577 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.517678 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.517749 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.517901 (+ 152us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.517942 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":94,"threads_started":1}
I20260805 22:43:06.518021 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.514080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:34493 (local address 127.0.0.1:50012)
0805 22:43:06.514518 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:06.514527 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:06.514560 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:06.514869 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:06.514873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:06.514875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:06.514948 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.514949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.515877 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.515878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.517011 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.517014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.517682 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.517683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.517766 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.517780 (+ 14us) client_negotiation.cc:769] Sending connection context
0805 22:43:06.517883 (+ 103us) client_negotiation.cc:240] Negotiation successful
0805 22:43:06.517943 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":2,"thread_start_us":141,"threads_started":1}
I20260805 22:43:06.518821 7310 tablet_service.cc:1467] Processing CreateTablet for tablet 7bb31b47363f45fd8e8dd84a8da00fa7 (DEFAULT_TABLE table=test-table [id=f369be5abd83413781ff433c11e2cc96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:43:06.519029 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.514076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:45357 (local address 127.0.0.1:43864)
0805 22:43:06.514280 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:06.514332 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:06.514407 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:06.515265 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:06.515266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:06.515267 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:06.515305 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.515306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.516661 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.516663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.517674 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.517677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.518770 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.518772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.518853 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.518857 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:43:06.518909 (+ 52us) client_negotiation.cc:240] Negotiation successful
0805 22:43:06.518953 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":60,"threads_started":1}
I20260805 22:43:06.519079 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.514203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.23.197.65:45357)
0805 22:43:06.515003 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.515007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.515017 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:43:06.515058 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.515060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.515077 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.515175 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:06.515394 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.516545 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.517779 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.518594 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.518673 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.518733 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.518946 (+ 213us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.518988 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":127,"threads_started":1}
I20260805 22:43:06.519220 7310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bb31b47363f45fd8e8dd84a8da00fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:06.519730 7118 tablet_service.cc:1467] Processing CreateTablet for tablet 7bb31b47363f45fd8e8dd84a8da00fa7 (DEFAULT_TABLE table=test-table [id=f369be5abd83413781ff433c11e2cc96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:43:06.520067 7118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bb31b47363f45fd8e8dd84a8da00fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:06.521529 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.514080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:38905 (local address 127.0.0.1:35898)
0805 22:43:06.514431 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:06.514440 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:06.514482 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:06.514951 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:06.514952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:06.514955 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:06.515019 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.515022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.516485 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.516486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.518716 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.518719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.521272 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.521273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.521344 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.521349 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:43:06.521409 (+ 60us) client_negotiation.cc:240] Negotiation successful
0805 22:43:06.521451 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":998,"thread_start_us":145,"threads_started":1}
I20260805 22:43:06.521555 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.514404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.23.197.67:38905)
0805 22:43:06.514692 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.514696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.514707 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:43:06.514757 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.514759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.514777 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.514856 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:06.515118 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.516366 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.518813 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.521153 (+ 2340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.521265 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.521327 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.521422 (+ 95us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.521467 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":109,"threads_started":1}
I20260805 22:43:06.522397 7504 tablet_service.cc:1467] Processing CreateTablet for tablet 7bb31b47363f45fd8e8dd84a8da00fa7 (DEFAULT_TABLE table=test-table [id=f369be5abd83413781ff433c11e2cc96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:43:06.522818 7504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bb31b47363f45fd8e8dd84a8da00fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:06.522790 7623 tablet_bootstrap.cc:492] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: Bootstrap starting.
I20260805 22:43:06.523521 7626 tablet_bootstrap.cc:492] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca: Bootstrap starting.
I20260805 22:43:06.523705 7623 tablet_bootstrap.cc:654] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:06.524093 7626 tablet_bootstrap.cc:654] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:06.524523 7626 log.cc:824] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:06.524509 7623 log.cc:824] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:06.525430 7623 tablet_bootstrap.cc:492] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: No bootstrap required, opened a new log
I20260805 22:43:06.525456 7626 tablet_bootstrap.cc:492] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca: No bootstrap required, opened a new log
I20260805 22:43:06.525503 7623 ts_tablet_manager.cc:1397] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 22:43:06.525530 7626 ts_tablet_manager.cc:1397] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:43:06.527244 7623 raft_consensus.cc:348] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } }
I20260805 22:43:06.527369 7623 raft_consensus.cc:374] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:06.527390 7623 raft_consensus.cc:729] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b9b5420e9cb42de8eaf55f9d70339fb, State: Initialized, Role: FOLLOWER
I20260805 22:43:06.527504 7623 consensus_queue.cc:260] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } }
I20260805 22:43:06.527642 7626 raft_consensus.cc:348] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } }
I20260805 22:43:06.527801 7626 raft_consensus.cc:374] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:06.527799 7623 ts_tablet_manager.cc:1428] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260805 22:43:06.527820 7402 heartbeater.cc:503] Master 127.23.197.126:40907 was elected leader, sending a full tablet report...
I20260805 22:43:06.527843 7626 raft_consensus.cc:729] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f664494af7b246c7b4a9133136ac1cca, State: Initialized, Role: FOLLOWER
I20260805 22:43:06.527982 7626 consensus_queue.cc:260] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } }
I20260805 22:43:06.528008 7629 tablet_bootstrap.cc:492] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: Bootstrap starting.
I20260805 22:43:06.528301 7626 ts_tablet_manager.cc:1428] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:43:06.528329 7214 heartbeater.cc:503] Master 127.23.197.126:40907 was elected leader, sending a full tablet report...
I20260805 22:43:06.529083 7629 tablet_bootstrap.cc:654] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:06.529729 7629 log.cc:824] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:06.530601 7629 tablet_bootstrap.cc:492] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: No bootstrap required, opened a new log
I20260805 22:43:06.530659 7629 ts_tablet_manager.cc:1397] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 22:43:06.531991 7629 raft_consensus.cc:348] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } }
I20260805 22:43:06.532115 7629 raft_consensus.cc:374] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:06.532181 7629 raft_consensus.cc:729] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed69eb945b6245d7a56247b0fd493b98, State: Initialized, Role: FOLLOWER
I20260805 22:43:06.532374 7629 consensus_queue.cc:260] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } }
I20260805 22:43:06.532688 7629 ts_tablet_manager.cc:1428] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:43:06.532703 7597 heartbeater.cc:503] Master 127.23.197.126:40907 was elected leader, sending a full tablet report...
W20260805 22:43:06.569340 7405 tablet.cc:2369] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:43:06.583403 7632 raft_consensus.cc:482] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:43:06.583598 7632 raft_consensus.cc:504] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } }
I20260805 22:43:06.584048 7632 leader_election.cc:288] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed69eb945b6245d7a56247b0fd493b98 (127.23.197.67:38905), 0b9b5420e9cb42de8eaf55f9d70339fb (127.23.197.66:34493)
I20260805 22:43:06.588112 7620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.584343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:59883 (local address 127.23.197.66:34493)
0805 22:43:06.584394 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.584396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.584408 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:43:06.584441 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.584442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.584457 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.584533 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:06.584806 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.585581 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.587116 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.587721 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.587800 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.587808 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.587996 (+ 188us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.588036 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:43:06.588112 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.584398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:48769 (local address 127.23.197.67:38905)
0805 22:43:06.584443 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.584445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.584605 (+ 160us) server_negotiation.cc:407] Connection header received
0805 22:43:06.584648 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.584649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.584656 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.584726 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:06.584916 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.585605 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.587116 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.587716 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.587799 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.587809 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.587996 (+ 187us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.588037 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:43:06.588133 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.584255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:34493 (local address 127.23.197.65:59883)
0805 22:43:06.584309 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:06.584324 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:06.584378 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:06.584563 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:06.584564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:06.584566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:06.584650 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.584652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.585744 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.585745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.587001 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.587004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.587812 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.587813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.587907 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.587909 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:43:06.587970 (+ 61us) client_negotiation.cc:240] Negotiation successful
0805 22:43:06.588036 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":37}
I20260805 22:43:06.588143 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.584288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:38905 (local address 127.23.197.65:48769)
0805 22:43:06.584494 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:06.584529 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:06.584581 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:06.584732 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:06.584732 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:06.584734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:06.584834 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.584836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.585682 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.585683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.587001 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:06.587004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.587812 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.587813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:06.587903 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.587908 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:43:06.587970 (+ 62us) client_negotiation.cc:240] Negotiation successful
0805 22:43:06.588034 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":99,"thread_start_us":136,"threads_started":1}
I20260805 22:43:06.588593 7333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bb31b47363f45fd8e8dd84a8da00fa7" candidate_uuid: "f664494af7b246c7b4a9133136ac1cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" is_pre_election: true
I20260805 22:43:06.588723 7532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bb31b47363f45fd8e8dd84a8da00fa7" candidate_uuid: "f664494af7b246c7b4a9133136ac1cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed69eb945b6245d7a56247b0fd493b98" is_pre_election: true
I20260805 22:43:06.588800 7333 raft_consensus.cc:2454] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f664494af7b246c7b4a9133136ac1cca in term 0.
I20260805 22:43:06.588891 7532 raft_consensus.cc:2454] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f664494af7b246c7b4a9133136ac1cca in term 0.
I20260805 22:43:06.589052 7052 leader_election.cc:302] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b9b5420e9cb42de8eaf55f9d70339fb, f664494af7b246c7b4a9133136ac1cca; no voters:
I20260805 22:43:06.589248 7632 raft_consensus.cc:2781] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:43:06.589314 7632 raft_consensus.cc:482] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:43:06.589339 7632 raft_consensus.cc:3037] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:06.590065 7632 raft_consensus.cc:504] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } }
I20260805 22:43:06.590221 7632 leader_election.cc:288] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [CANDIDATE]: Term 1 election: Requested vote from peers ed69eb945b6245d7a56247b0fd493b98 (127.23.197.67:38905), 0b9b5420e9cb42de8eaf55f9d70339fb (127.23.197.66:34493)
I20260805 22:43:06.590372 7532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bb31b47363f45fd8e8dd84a8da00fa7" candidate_uuid: "f664494af7b246c7b4a9133136ac1cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed69eb945b6245d7a56247b0fd493b98"
I20260805 22:43:06.590384 7333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bb31b47363f45fd8e8dd84a8da00fa7" candidate_uuid: "f664494af7b246c7b4a9133136ac1cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb"
I20260805 22:43:06.590456 7333 raft_consensus.cc:3037] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:06.590456 7532 raft_consensus.cc:3037] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:06.591421 7333 raft_consensus.cc:2454] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f664494af7b246c7b4a9133136ac1cca in term 1.
I20260805 22:43:06.591421 7532 raft_consensus.cc:2454] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f664494af7b246c7b4a9133136ac1cca in term 1.
I20260805 22:43:06.591595 7054 leader_election.cc:302] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed69eb945b6245d7a56247b0fd493b98, f664494af7b246c7b4a9133136ac1cca; no voters:
I20260805 22:43:06.591702 7632 raft_consensus.cc:2781] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:43:06.591893 7632 raft_consensus.cc:686] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 1 LEADER]: Becoming Leader. State: Replica: f664494af7b246c7b4a9133136ac1cca, State: Running, Role: LEADER
I20260805 22:43:06.592007 7632 consensus_queue.cc:237] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } }
I20260805 22:43:06.592954 6930 catalog_manager.cc:5495] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca reported cstate change: term changed from 0 to 1, leader changed from <none> to f664494af7b246c7b4a9133136ac1cca (127.23.197.65). New cstate: current_term: 1 leader_uuid: "f664494af7b246c7b4a9133136ac1cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f664494af7b246c7b4a9133136ac1cca" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 45357 } health_report { overall_health: HEALTHY } } }
I20260805 22:43:06.610975 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.608563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43878 (local address 127.23.197.65:45357)
0805 22:43:06.608632 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:06.608633 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:06.608642 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:43:06.608684 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:06.608685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:06.608707 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:06.608820 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:43:06.608968 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.609438 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.610199 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:06.610387 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:06.610441 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:06.610576 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:43:06.610680 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:43:06.610744 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:06.610811 (+ 67us) server_negotiation.cc:299] Negotiation successful
0805 22:43:06.610830 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 22:43:06.613226 7532 raft_consensus.cc:1261] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [term 1 FOLLOWER]: Refusing update from remote peer f664494af7b246c7b4a9133136ac1cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:43:06.613224 7333 raft_consensus.cc:1261] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [term 1 FOLLOWER]: Refusing update from remote peer f664494af7b246c7b4a9133136ac1cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:43:06.613631 7632 consensus_queue.cc:1035] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed69eb945b6245d7a56247b0fd493b98" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:43:06.613807 7639 consensus_queue.cc:1035] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b9b5420e9cb42de8eaf55f9d70339fb" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 34493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:43:06.616839 7645 mvcc.cc:205] Tried to move back new op lower bound from 7315332245964386304 to 7315332245881614336. Current Snapshot: MvccSnapshot[applied={T|T < 7315332245964386304}]
I20260805 22:43:06.617957 7642 mvcc.cc:205] Tried to move back new op lower bound from 7315332245964386304 to 7315332245881614336. Current Snapshot: MvccSnapshot[applied={T|T < 7315332245964386304}]
I20260805 22:43:06.620906 6929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:43:06.621028 6929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:43:06.621909 6929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90c9632385cf4dcbba98f2dc276e0b35: Sending DeleteTablet for 3 replicas of tablet 7bb31b47363f45fd8e8dd84a8da00fa7
I20260805 22:43:06.622280 7118 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bb31b47363f45fd8e8dd84a8da00fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:43:06 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:43864
I20260805 22:43:06.622311 7504 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bb31b47363f45fd8e8dd84a8da00fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:43:06 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:35898
I20260805 22:43:06.622550 7658 tablet_replica.cc:330] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: stopping tablet replica
I20260805 22:43:06.622655 7658 raft_consensus.cc:2229] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:43:06.622776 7654 tablet_replica.cc:330] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca: stopping tablet replica
I20260805 22:43:06.622787 7658 raft_consensus.cc:2258] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:43:06.622913 7654 raft_consensus.cc:2229] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:43:06.623068 7654 raft_consensus.cc:2258] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:43:06.623293 7658 ts_tablet_manager.cc:1905] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:43:06.623708 7654 ts_tablet_manager.cc:1905] T 7bb31b47363f45fd8e8dd84a8da00fa7 P f664494af7b246c7b4a9133136ac1cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:43:06.624305 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 7025
I20260805 22:43:06.625147 7658 ts_tablet_manager.cc:1918] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:43:06.625226 7658 log.cc:1196] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/7bb31b47363f45fd8e8dd84a8da00fa7
I20260805 22:43:06.625564 7658 ts_tablet_manager.cc:1939] T 7bb31b47363f45fd8e8dd84a8da00fa7 P ed69eb945b6245d7a56247b0fd493b98: Deleting consensus metadata
I20260805 22:43:06.626068 6912 catalog_manager.cc:4841] TS ed69eb945b6245d7a56247b0fd493b98 (127.23.197.67:38905): tablet 7bb31b47363f45fd8e8dd84a8da00fa7 (table test-table [id=f369be5abd83413781ff433c11e2cc96]) successfully deleted
I20260805 22:43:06.629711 7310 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bb31b47363f45fd8e8dd84a8da00fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:43:06 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:50012
I20260805 22:43:06.631031 7659 tablet_replica.cc:330] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: stopping tablet replica
I20260805 22:43:06.631179 7659 raft_consensus.cc:2229] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:43:06.631313 7659 raft_consensus.cc:2258] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:43:06.631814 7659 ts_tablet_manager.cc:1905] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:43:06.633955 7659 ts_tablet_manager.cc:1918] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:43:06.634057 7659 log.cc:1196] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/7bb31b47363f45fd8e8dd84a8da00fa7
I20260805 22:43:06.634398 7659 ts_tablet_manager.cc:1939] T 7bb31b47363f45fd8e8dd84a8da00fa7 P 0b9b5420e9cb42de8eaf55f9d70339fb: Deleting consensus metadata
I20260805 22:43:06.635143 6910 catalog_manager.cc:4841] TS 0b9b5420e9cb42de8eaf55f9d70339fb (127.23.197.66:34493): tablet 7bb31b47363f45fd8e8dd84a8da00fa7 (table test-table [id=f369be5abd83413781ff433c11e2cc96]) successfully deleted
W20260805 22:43:06.636695 6911 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:45357 (error 108)
I20260805 22:43:06.637185 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 7221
I20260805 22:43:06.637424 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:06.637217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:45357 (local address 127.0.0.1:43892)
0805 22:43:06.637327 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:06.637382 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:45357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260805 22:43:06.641496 6911 catalog_manager.cc:4584] TS f664494af7b246c7b4a9133136ac1cca (127.23.197.65:45357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7bb31b47363f45fd8e8dd84a8da00fa7: Network error: Client connection negotiation failed: client connection to 127.23.197.65:45357: connect: Connection refused (error 111)
I20260805 22:43:06.645269 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 7410
I20260805 22:43:06.656694 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 6889
2026-08-05T22:43:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2225 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:43:06 dist-test-slave-g170 krb5kdc[7663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:43:06 dist-test-slave-g170 krb5kdc[7663](info): set up 2 sockets
Aug 05 22:43:06 dist-test-slave-g170 krb5kdc[7663](info): commencing operation
krb5kdc: starting...
W20260805 22:43:07.989807 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:43:08 dist-test-slave-g170 krb5kdc[7663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:43:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:43:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.197.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.197.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/rangeradmin_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/rangeradmin_127.23.197.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.197.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.197.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/rangerlookup_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/rangerlookup_127.23.197.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.197.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.197.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/HTTP_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/HTTP_127.23.197.84@KRBTEST.COM.keytab.
I20260805 22:43:08.188369 24341 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskHuFw91/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/postgres -l logfile start
2026-08-05 22:43:10.940 UTC [7711] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-05 22:43:10.940 UTC [7711] LOG: listening on IPv4 address "127.23.197.84", port 42125
2026-08-05 22:43:10.944 UTC [7711] LOG: listening on Unix socket "/tmp/.s.PGSQL.42125"
2026-08-05 22:43:10.967 UTC [7715] LOG: database system was shut down at 2026-08-05 22:43:09 UTC
2026-08-05 22:43:10.971 UTC [7711] LOG: database system is ready to accept connections
I20260805 22:43:12.198689 24341 mini_postgres.cc:96] Postgres bound to 42125
2026-08-05 22:43:12.206 UTC [7728] FATAL: database "slave" does not exist
127.23.197.84:42125 - accepting connections
I20260805 22:43:12.207094 24341 mini_ranger.cc:161] Starting Ranger...
I20260805 22:43:12.225564 24341 mini_ranger.cc:84] Created miniranger Postgres user
I20260805 22:43:12.750910 24341 mini_ranger.cc:87] Created ranger Postgres database
I20260805 22:43:12.751012 24341 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-admin
2026-08-05 22:43:13,053 [I] DB FLAVOR :POSTGRES
2026-08-05 22:43:13,054 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 22:43:13,054 [I] Checking connection..
2026-08-05 22:43:13,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-05 22:43:13,319 [I] Checking connection passed.
2026-08-05 22:43:13,319 [I] --------- Verifying version history table ---------
2026-08-05 22:43:13,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 22:43:13,567 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 22:43:13,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 22:43:13,832 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 22:43:13,832 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-05 22:43:13,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-05 22:43:14.104 UTC [7822] WARNING: there is no transaction in progress
2026-08-05 22:43:14,113 [I] create_dbversion_catalog.sql file imported successfully
2026-08-05 22:43:14,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 22:43:14,410 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-05 22:43:14,410 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 22:43:14,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-05 22:43:14,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-05 22:43:14,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-g170.c.gcp-upstream.internal','N') ;"
2026-08-05 22:43:15,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 22:43:15,502 [I] Table x_portal_user does not exist in database ranger
2026-08-05 22:43:15,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 22:43:15,777 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-05 22:43:15,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 22:43:16,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 22:43:16,284 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-05 22:43:16,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-05 22:43:17.397 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.415 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.432 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.450 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.468 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.492 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.550 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.557 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.573 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.585 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.596 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.611 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.621 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.628 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.636 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:17.643 UTC [7979] WARNING: there is no transaction in progress
2026-08-05 22:43:18,369 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-05 22:43:18,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-05 22:43:18,635 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-05 22:43:18,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 22:43:18,901 [I] Table x_portal_user already exists in database 'ranger'
2026-08-05 22:43:18,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 22:43:19,142 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-05 22:43:19,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 22:43:19,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 22:43:19,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-g170.c.gcp-upstream.internal';"
2026-08-05 22:43:19,975 [I] CORE_DB_SCHEMA import status has been updated
2026-08-05 22:43:19,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-05 22:43:20,241 [I] DB_PATCHES have already been applied
I20260805 22:43:20.247447 24341 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-admin:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskHuFw91/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260805 22:43:20.247517 24341 mini_ranger.cc:193] Using host: 127.23.197.84
I20260805 22:43:20.251653 24341 mini_ranger.cc:227] Ranger admin URL: http://127.23.197.84:45271
Aug 05, 2026 10:43:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 10:43:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 10:43:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-admin/ews/webapp
Aug 05, 2026 10:43:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 05, 2026 10:43:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 05, 2026 10:43:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-admin/ews/webapp].
Aug 05, 2026 10:43:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260805 22:43:21.597640 24341 mini_ranger.cc:160] Time spent starting Ranger: real 9.391s user 0.000s sys 0.012s
Aug 05, 2026 10:43:41 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 05, 2026 10:43:41 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 05, 2026 10:43:41 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 05, 2026 10:43:41 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 05, 2026 10:43:42 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260805 22:43:43.468911 24341 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.197.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.197.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/rangerkms_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/rangerkms_127.23.197.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.197.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.197.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.197.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/HTTP_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.197.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/HTTP_127.23.197.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 05 22:43:43 dist-test-slave-g170 krb5kdc[7663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969823, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260805 22:43:43.561218 24341 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260805 22:43:43.572546 24341 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260805 22:43:44.576010 24341 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260805 22:43:44.576102 24341 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-kms
I20260805 22:43:44.576121 24341 mini_ranger_kms.cc:227] Using postgres at 127.23.197.84:42125
2026-08-05 22:43:44,908 [I] DB FLAVOR :POSTGRES
2026-08-05 22:43:44,908 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 22:43:44,908 [I] Checking connection
2026-08-05 22:43:44,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-05 22:43:45,197 [I] connection success
2026-08-05 22:43:45,197 [I] --------- Verifying Ranger DB tables ---------
2026-08-05 22:43:45,197 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-05 22:43:45,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-05 22:43:45,505 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-05 22:43:45,506 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 22:43:45,506 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-05 22:43:45,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:42125/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-05 22:43:45,776 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260805 22:43:46.260408 24341 mini_ranger_kms.cc:310] Created kms service
I20260805 22:43:46.453619 24341 mini_ranger_kms.cc:326] Created kudu user
I20260805 22:43:46.497387 24341 mini_ranger_kms.cc:343] Created rangerkms user
Aug 05 22:43:46 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969823, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:43:46.921221 24341 mini_ranger_kms.cc:384] Added ranger policy
I20260805 22:43:46.921339 24341 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-kms:/tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260805 22:43:46.921360 24341 mini_ranger_kms.cc:240] Using host: 127.23.197.84
I20260805 22:43:46.925529 24341 mini_ranger_kms.cc:276] Ranger KMS PID: 8248
I20260805 22:43:46.925688 24341 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.197.84:45275
Aug 05, 2026 10:43:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 10:43:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 10:43:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 05, 2026 10:43:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 05, 2026 10:43:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-kms/ews/webapp].
Aug 05, 2026 10:43:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 05 22:43:49 dist-test-slave-g170 krb5kdc[7663](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785969829, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.197.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:49.901724 24341 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.341s user 0.011s sys 0.291s
I20260805 22:43:49.901875 24341 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260805 22:43:49.901975 24341 mini_ranger_kms.cc:401] 127.23.197.84:45275/kms/v1/keys
Aug 05 22:43:50 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969829, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.197.84@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
Aug 05 22:43:50 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969829, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.197.84@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
Aug 05 22:43:52 dist-test-slave-g170 krb5kdc[7663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:43:52.767730 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38413
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:38413
--ranger_config_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:45275/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:43:52.855862 8328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:52.856084 8328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:52.856108 8328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:52.857398 8328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:43:52.857450 8328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:52.857465 8328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:52.857476 8328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:43:52.857488 8328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:43:52.859216 8328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.197.84:45275/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:38413
--ranger_config_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:52.859457 8328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:52.859647 8328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:52.865310 8335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:52.865319 8342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:52.865583 8328 server_base.cc:1034] running on GCE node
W20260805 22:43:52.865319 8337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:52.866031 8328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:52.866405 8328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:52.867565 8328 hybrid_clock.cc:648] HybridClock initialized: now 1785969832867530 us; error 64 us; skew 500 ppm
Aug 05 22:43:52 dist-test-slave-g170 krb5kdc[7663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969832, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:52.873692 8328 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:43:52.874123 8328 webserver.cc:460] Webserver started at http://127.23.197.126:41147/ using document root <none> and password file <none>
I20260805 22:43:52.874330 8328 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:52.874367 8328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:52.874462 8328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 22:43:52 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969832, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:43:52.937902 8328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "c7ce8496bf854736b79e3e6ece3a47e8"
format_stamp: "Formatted at 2026-08-05 22:43:52 on dist-test-slave-g170"
server_key: "4a5edc166a0394271d16b65322b9d138"
server_key_iv: "db8519dc845ef9efa1ced7d55fcdf6c0"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:52.938480 8328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "c7ce8496bf854736b79e3e6ece3a47e8"
format_stamp: "Formatted at 2026-08-05 22:43:52 on dist-test-slave-g170"
server_key: "4a5edc166a0394271d16b65322b9d138"
server_key_iv: "db8519dc845ef9efa1ced7d55fcdf6c0"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:52.940577 8328 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260805 22:43:52.955659 8350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:52.955962 8328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:43:52.956080 8328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "c7ce8496bf854736b79e3e6ece3a47e8"
format_stamp: "Formatted at 2026-08-05 22:43:52 on dist-test-slave-g170"
server_key: "4a5edc166a0394271d16b65322b9d138"
server_key_iv: "db8519dc845ef9efa1ced7d55fcdf6c0"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:52.956210 8328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:52.967993 8328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:52.970249 8328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:52.970425 8328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:52.975586 8328 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:38413
I20260805 22:43:52.975613 8431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:38413 every 8 connection(s)
I20260805 22:43:52.976171 8328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:43:52.976979 8432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:52.979303 8432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8: Bootstrap starting.
I20260805 22:43:52.979925 8432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:52.980239 8432 log.cc:824] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:52.981083 8432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8: No bootstrap required, opened a new log
I20260805 22:43:52.983213 8432 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce8496bf854736b79e3e6ece3a47e8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38413 } }
I20260805 22:43:52.983358 8432 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:52.983382 8432 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ce8496bf854736b79e3e6ece3a47e8, State: Initialized, Role: FOLLOWER
I20260805 22:43:52.983536 8432 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce8496bf854736b79e3e6ece3a47e8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38413 } }
I20260805 22:43:52.983615 8432 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:43:52.983639 8432 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:43:52.983708 8432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:52.984344 8432 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce8496bf854736b79e3e6ece3a47e8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38413 } }
I20260805 22:43:52.984434 8432 leader_election.cc:302] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7ce8496bf854736b79e3e6ece3a47e8; no voters:
I20260805 22:43:52.984679 8432 leader_election.cc:288] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:43:52.984700 8439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:43:52.984856 8439 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [term 1 LEADER]: Becoming Leader. State: Replica: c7ce8496bf854736b79e3e6ece3a47e8, State: Running, Role: LEADER
I20260805 22:43:52.984954 8439 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce8496bf854736b79e3e6ece3a47e8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38413 } }
I20260805 22:43:52.984983 8432 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:43:52.985028 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 8328
I20260805 22:43:52.985147 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:43:52.985256 8439 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7ce8496bf854736b79e3e6ece3a47e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce8496bf854736b79e3e6ece3a47e8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38413 } } }
I20260805 22:43:52.985316 8439 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [sys.catalog]: This master's current role is: LEADER
I20260805 22:43:52.985333 24341 external_mini_cluster.cc:1428] Setting key 6074f63c4029be0d373c9c790893fb12
I20260805 22:43:52.985424 8439 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7ce8496bf854736b79e3e6ece3a47e8. Latest consensus state: current_term: 1 leader_uuid: "c7ce8496bf854736b79e3e6ece3a47e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce8496bf854736b79e3e6ece3a47e8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38413 } } }
I20260805 22:43:52.985482 8439 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8 [sys.catalog]: This master's current role is: LEADER
I20260805 22:43:52.986887 8432 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260805 22:43:52.990442 8445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:43:52.991784 8445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:43:52.994783 8445 catalog_manager.cc:1347] Generated new cluster ID: c2cc331d02214020a9173bdada91c8ef
I20260805 22:43:52.994903 8445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:43:52 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:43:52.999680 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:52.985774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.23.197.126:38413)
0805 22:43:52.985979 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:52.985989 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:52.986150 (+ 161us) server_negotiation.cc:407] Connection header received
0805 22:43:52.986322 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:52.986330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:52.986401 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:52.986614 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:52.987359 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:52.991137 (+ 3778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:52.992290 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:52.993264 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:52.993406 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:52.996183 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:52.996188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:52.996188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:52.996200 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:52.998104 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:52.998504 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:52.998507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:52.998507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:52.998625 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:52.998811 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:52.998812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:52.998812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:52.999093 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:52.999218 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:52.999343 (+ 125us) server_negotiation.cc:299] Negotiation successful
0805 22:43:52.999462 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":81,"threads_started":1}
I20260805 22:43:53.022302 8445 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:43:53.023151 8445 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:43:53.027410 8445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8: Generated new TSK 0
I20260805 22:43:53.027688 8445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:43:53.755772 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:45275/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38413
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:43:53.869002 8508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:53.869225 8508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:53.869257 8508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:53.870630 8508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:53.870710 8508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:53.870757 8508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:43:53.872521 8508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.197.84:45275/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:53.872879 8508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:53.873219 8508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:53.879462 8517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:53.879655 8508 server_base.cc:1034] running on GCE node
W20260805 22:43:53.879479 8518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:53.879477 8520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:53.880154 8508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:53.880491 8508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:53.881878 8508 hybrid_clock.cc:648] HybridClock initialized: now 1785969833881844 us; error 54 us; skew 500 ppm
Aug 05 22:43:53 dist-test-slave-g170 krb5kdc[7663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969833, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:53.885361 8508 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:43:53.886023 8508 webserver.cc:460] Webserver started at http://127.23.197.65:46419/ using document root <none> and password file <none>
I20260805 22:43:53.886325 8508 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:53.886409 8508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:53.886579 8508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 22:43:53 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969833, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:43:53.902125 8508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f"
format_stamp: "Formatted at 2026-08-05 22:43:53 on dist-test-slave-g170"
server_key: "25631341afeee37e58489a97ea4357a8"
server_key_iv: "c784a37950b016c44dde8f94c6d2db3e"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:53.902653 8508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f"
format_stamp: "Formatted at 2026-08-05 22:43:53 on dist-test-slave-g170"
server_key: "25631341afeee37e58489a97ea4357a8"
server_key_iv: "c784a37950b016c44dde8f94c6d2db3e"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:53.904708 8508 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260805 22:43:53.919246 8531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:53.919536 8508 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:43:53.919656 8508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f"
format_stamp: "Formatted at 2026-08-05 22:43:53 on dist-test-slave-g170"
server_key: "25631341afeee37e58489a97ea4357a8"
server_key_iv: "c784a37950b016c44dde8f94c6d2db3e"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:53.919749 8508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:53.936141 8508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:53.939747 8508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:53.939994 8508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:53.940307 8508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:53.940755 8508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:53.940886 8508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:53.940953 8508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:53.940990 8508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:53.947940 8508 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:40787
I20260805 22:43:53.948273 8712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:40787 every 8 connection(s)
I20260805 22:43:53.948534 8508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
Aug 05 22:43:53 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969833, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:43:53.958187 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 8508
I20260805 22:43:53.958304 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:43:53.958551 24341 external_mini_cluster.cc:1428] Setting key 0f49396b85c4c9547262b0bdc0697d82
I20260805 22:43:53.965325 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:53.949403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38413 (local address 127.23.197.65:42733)
0805 22:43:53.949917 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:53.949970 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:53.950648 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:53.951122 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:53.951127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:53.951402 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:53.951696 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:53.951704 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:53.952691 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:53.952692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:53.954046 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:53.954049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:53.954625 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:53.954626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:53.954790 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:53.955448 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:53.955459 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:53.957346 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:53.963209 (+ 5863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:53.963220 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:53.963224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:53.963590 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:53.963865 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:53.963866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:53.963867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:53.964079 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:53.964455 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:53.964463 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:53.964655 (+ 192us) client_negotiation.cc:769] Sending connection context
0805 22:43:53.964850 (+ 195us) client_negotiation.cc:240] Negotiation successful
0805 22:43:53.965050 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":87,"threads_started":1}
I20260805 22:43:53.965826 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:53.949746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:42733 (local address 127.23.197.126:38413)
0805 22:43:53.950871 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:53.950874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:53.950883 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:43:53.950910 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:53.950910 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:53.950923 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:53.950991 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:53.951829 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:53.952570 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:53.954153 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:53.954466 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:53.954544 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:53.957463 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:53.957465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:53.957466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:53.957475 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:53.963097 (+ 5622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:53.963701 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:53.963703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:53.963703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:53.963781 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:53.964173 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:53.964174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:53.964175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:53.964354 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:53.964424 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:53.965607 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0805 22:43:53.965721 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1089,"thread_start_us":74,"threads_started":1}
I20260805 22:43:53.966236 8715 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38413
I20260805 22:43:53.966341 8715 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:53.966549 8715 heartbeater.cc:511] Master 127.23.197.126:38413 requested a full tablet report, sending...
I20260805 22:43:53.967108 8378 ts_manager.cc:194] Registered new tserver with Master: c8b82bb1bd3c4cf4b25a70f39aa3897f (127.23.197.65:40787)
I20260805 22:43:53.968052 8378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:42733
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:43:54.000685 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:45275/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38413
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:43:54.125936 8723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:54.126348 8723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:54.126457 8723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:54.128222 8723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:54.128454 8723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:54.128545 8723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:43:54.130780 8723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.197.84:45275/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:54.131266 8723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:54.131582 8723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:54.138525 8733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:54.139070 8734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:54.139635 8723 server_base.cc:1034] running on GCE node
W20260805 22:43:54.139905 8738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:54.140190 8723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:54.140470 8723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:54.141758 8723 hybrid_clock.cc:648] HybridClock initialized: now 1785969834141727 us; error 46 us; skew 500 ppm
Aug 05 22:43:54 dist-test-slave-g170 krb5kdc[7663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:54.145323 8723 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:43:54.145954 8723 webserver.cc:460] Webserver started at http://127.23.197.66:32773/ using document root <none> and password file <none>
I20260805 22:43:54.146515 8723 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:54.146816 8723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:54.147012 8723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 22:43:54 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
Aug 05 22:43:54 dist-test-slave-g170 krb5kdc[7663](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785969834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:54.162436 8723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "c9def1dc42ab4a3788db1f171bc93adc"
format_stamp: "Formatted at 2026-08-05 22:43:54 on dist-test-slave-g170"
server_key: "2bc688945d1b21999f9575954db3e536"
server_key_iv: "834a10cb860d56a9dd7270122faf9b56"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:54.163035 8723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "c9def1dc42ab4a3788db1f171bc93adc"
format_stamp: "Formatted at 2026-08-05 22:43:54 on dist-test-slave-g170"
server_key: "2bc688945d1b21999f9575954db3e536"
server_key_iv: "834a10cb860d56a9dd7270122faf9b56"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:54.166555 8723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260805 22:43:54.188823 8748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:54.189095 8723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260805 22:43:54.189193 8723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "c9def1dc42ab4a3788db1f171bc93adc"
format_stamp: "Formatted at 2026-08-05 22:43:54 on dist-test-slave-g170"
server_key: "2bc688945d1b21999f9575954db3e536"
server_key_iv: "834a10cb860d56a9dd7270122faf9b56"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:54.189296 8723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:54.207958 8723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:54.210512 8723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:54.210736 8723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:54.210994 8723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:54.211410 8723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:54.211508 8723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:54.211581 8723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:54.211639 8723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:54.219449 8723 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:38903
I20260805 22:43:54.219988 8723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:43:54.222483 8921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:38903 every 8 connection(s)
I20260805 22:43:54.229132 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 8723
I20260805 22:43:54.229252 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:43:54.229442 24341 external_mini_cluster.cc:1428] Setting key 01eca2be77310bb3b5bf5fbf6799cf1c
Aug 05 22:43:54 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:43:54.234097 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.222363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:46691 (local address 127.23.197.126:38413)
0805 22:43:54.222409 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:54.222412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:54.222953 (+ 541us) server_negotiation.cc:407] Connection header received
0805 22:43:54.223660 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:54.223661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:54.223687 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:54.223721 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:54.224637 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.225542 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.227161 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.227479 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.227525 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.230329 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:54.230330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:54.230330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:54.230338 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:54.232244 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:54.232855 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:54.232856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:54.232857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:54.232921 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:54.233214 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:54.233215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:54.233215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:54.233361 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:54.233419 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:54.233843 (+ 424us) server_negotiation.cc:299] Negotiation successful
0805 22:43:54.233948 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:43:54.234614 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.222070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38413 (local address 127.23.197.66:46691)
0805 22:43:54.222825 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:54.222866 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:54.223525 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:54.223913 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:54.223917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:54.224244 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:54.224519 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.224525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.225728 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.225729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.227012 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.227015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.227617 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.227617 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.227769 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.228399 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:54.228410 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:54.230214 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:54.232346 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:54.232352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:54.232355 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:54.232706 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:54.232997 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:54.232998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:54.232998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:54.233136 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:54.233470 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:54.233476 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:54.233666 (+ 190us) client_negotiation.cc:769] Sending connection context
0805 22:43:54.234225 (+ 559us) client_negotiation.cc:240] Negotiation successful
0805 22:43:54.234402 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":99,"threads_started":1}
I20260805 22:43:54.235146 8922 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38413
I20260805 22:43:54.235253 8922 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:54.235445 8922 heartbeater.cc:511] Master 127.23.197.126:38413 requested a full tablet report, sending...
I20260805 22:43:54.235790 8378 ts_manager.cc:194] Registered new tserver with Master: c9def1dc42ab4a3788db1f171bc93adc (127.23.197.66:38903)
I20260805 22:43:54.236299 8378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:46691
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:43:54.281473 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:45275/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38413
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:43:54.410928 8929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:43:54.411281 8929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:43:54.411347 8929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:43:54.412971 8929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:43:54.413074 8929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:43:54.413141 8929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:43:54.415187 8929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.197.84:45275/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:43:54.415594 8929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:43:54.415928 8929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:43:54.422816 8936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:43:54.423111 8938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:54.424247 8929 server_base.cc:1034] running on GCE node
W20260805 22:43:54.425165 8934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:43:54.425472 8929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:43:54.425745 8929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:43:54.426918 8929 hybrid_clock.cc:648] HybridClock initialized: now 1785969834426902 us; error 28 us; skew 500 ppm
Aug 05 22:43:54 dist-test-slave-g170 krb5kdc[7663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:43:54.430454 8929 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:43:54.430999 8929 webserver.cc:460] Webserver started at http://127.23.197.67:38857/ using document root <none> and password file <none>
I20260805 22:43:54.431311 8929 fs_manager.cc:359] Metadata directory not provided
I20260805 22:43:54.431371 8929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:43:54.431506 8929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 22:43:54 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:43:54.454509 8929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "c9403671dc4b4da292df571f3c9d35cd"
format_stamp: "Formatted at 2026-08-05 22:43:54 on dist-test-slave-g170"
server_key: "7070f7c6d8ed37172f6cdc0b6d56b1f6"
server_key_iv: "44b5d94fa1104a0df7c55789930303ff"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:54.455169 8929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "c9403671dc4b4da292df571f3c9d35cd"
format_stamp: "Formatted at 2026-08-05 22:43:54 on dist-test-slave-g170"
server_key: "7070f7c6d8ed37172f6cdc0b6d56b1f6"
server_key_iv: "44b5d94fa1104a0df7c55789930303ff"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:54.460558 8929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260805 22:43:54.477064 8946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:54.477360 8929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260805 22:43:54.477511 8929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "c9403671dc4b4da292df571f3c9d35cd"
format_stamp: "Formatted at 2026-08-05 22:43:54 on dist-test-slave-g170"
server_key: "7070f7c6d8ed37172f6cdc0b6d56b1f6"
server_key_iv: "44b5d94fa1104a0df7c55789930303ff"
server_key_version: "kuduclusterkey@0"
I20260805 22:43:54.477681 8929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:43:54.502895 8929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:43:54.506314 8929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:43:54.506646 8929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:43:54.507087 8929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:43:54.507751 8929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:43:54.507865 8929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:54.507987 8929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:43:54.508070 8929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:43:54.517374 8929 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:33025
I20260805 22:43:54.517952 8929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:43:54.524881 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 8929
I20260805 22:43:54.525015 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:43:54.520221 9115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:33025 every 8 connection(s)
I20260805 22:43:54.525203 24341 external_mini_cluster.cc:1428] Setting key 5a5addecf2c71d3d0546f621477c9bdc
Aug 05 22:43:54 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:43:54.532452 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.520940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:41039 (local address 127.23.197.126:38413)
0805 22:43:54.520993 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:54.520995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:54.521474 (+ 479us) server_negotiation.cc:407] Connection header received
0805 22:43:54.522222 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:54.522224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:54.522238 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:54.522273 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:54.523187 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.523945 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.525886 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.526182 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.526230 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.528990 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:54.528992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:54.528992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:54.528999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:54.530666 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:54.531197 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:54.531198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:54.531198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:54.531262 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:54.531562 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:54.531563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:54.531563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:54.531722 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:54.531773 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:54.532229 (+ 456us) server_negotiation.cc:299] Negotiation successful
0805 22:43:54.532325 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:43:54.533360 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.520929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38413 (local address 127.23.197.67:41039)
0805 22:43:54.521331 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:54.521379 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:54.522073 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:54.522473 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:54.522477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:54.522767 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:43:54.523053 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.523060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.524088 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.524088 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.525783 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.525786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.526328 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.526329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.526469 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.527071 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:43:54.527082 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:43:54.528877 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:43:54.530759 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:54.530766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:54.530769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:54.531091 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:54.531339 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:43:54.531339 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:43:54.531340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:43:54.531475 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:43:54.531829 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:43:54.531837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:43:54.532028 (+ 191us) client_negotiation.cc:769] Sending connection context
0805 22:43:54.532846 (+ 818us) client_negotiation.cc:240] Negotiation successful
0805 22:43:54.533048 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":90,"threads_started":1}
I20260805 22:43:54.534220 9116 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38413
I20260805 22:43:54.534332 9116 heartbeater.cc:464] Registering TS with master...
I20260805 22:43:54.534562 9116 heartbeater.cc:511] Master 127.23.197.126:38413 requested a full tablet report, sending...
I20260805 22:43:54.535104 8378 ts_manager.cc:194] Registered new tserver with Master: c9403671dc4b4da292df571f3c9d35cd (127.23.197.67:33025)
I20260805 22:43:54.535638 8378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:41039
I20260805 22:43:54.537348 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:43:54.547117 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.539230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44468 (local address 127.23.197.126:38413)
0805 22:43:54.539379 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:54.539381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:54.539444 (+ 63us) server_negotiation.cc:407] Connection header received
0805 22:43:54.539516 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:54.539517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:54.539528 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:54.539637 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:43:54.540486 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.541204 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.542685 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.542891 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.542981 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.544406 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:43:54.544408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:43:54.544409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:43:54.544416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:43:54.545902 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:54.546275 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:54.546276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:54.546276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:54.546338 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:43:54.546555 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:43:54.546556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:43:54.546557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:43:54.546674 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:43:54.546782 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:54.546926 (+ 144us) server_negotiation.cc:299] Negotiation successful
0805 22:43:54.547009 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260805 22:43:54.558950 8378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:43:54.561369 8378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:43:54.585902 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.566222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:40787 (local address 127.0.0.1:54386)
0805 22:43:54.568070 (+ 1848us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:54.568079 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:54.568114 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:54.568562 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:54.568566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:54.568568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:54.568637 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.568639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.571180 (+ 2541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.571181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.577818 (+ 6637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.577821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.584329 (+ 6508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.584330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.584434 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.584456 (+ 22us) client_negotiation.cc:769] Sending connection context
0805 22:43:54.584532 (+ 76us) client_negotiation.cc:240] Negotiation successful
0805 22:43:54.584598 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1835,"thread_start_us":103,"threads_started":1}
I20260805 22:43:54.587781 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.566415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:38903 (local address 127.0.0.1:56810)
0805 22:43:54.568835 (+ 2420us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:54.568845 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:54.568893 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:54.571978 (+ 3085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:54.571979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:54.571981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:54.572017 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.572019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.581620 (+ 9601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.581622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.585082 (+ 3460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.585085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.586638 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.586639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.586731 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.586737 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 22:43:54.586794 (+ 57us) client_negotiation.cc:240] Negotiation successful
0805 22:43:54.587705 (+ 911us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2402,"thread_start_us":68,"threads_started":1}
I20260805 22:43:54.588117 9148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.566899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.23.197.66:38903)
0805 22:43:54.568512 (+ 1613us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:54.568516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:54.568953 (+ 437us) server_negotiation.cc:407] Connection header received
0805 22:43:54.569001 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:54.569003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:54.569019 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:54.569091 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:54.572104 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.581524 (+ 9420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.585218 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.586437 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.586515 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.586600 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:54.587986 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0805 22:43:54.588029 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1573,"thread_start_us":73,"threads_started":1}
I20260805 22:43:54.589164 8818 tablet_service.cc:1467] Processing CreateTablet for tablet 1931e64fd5fc41cd9f4818dd179bc7a2 (DEFAULT_TABLE table=test-table [id=d510aee5e6a64ba5a35b036be0dd001d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:43:54.589593 8818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1931e64fd5fc41cd9f4818dd179bc7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:54.595002 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.566757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.23.197.65:40787)
0805 22:43:54.568224 (+ 1467us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:54.568230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:54.568241 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:43:54.568277 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:54.568279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:54.568297 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:54.568371 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:54.568742 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.571071 (+ 2329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.583148 (+ 12077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.584002 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.584082 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.594290 (+ 10208us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:54.594329 (+ 39us) server_negotiation.cc:299] Negotiation successful
0805 22:43:54.594678 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1428,"thread_start_us":63,"threads_started":1}
I20260805 22:43:54.596112 8606 tablet_service.cc:1467] Processing CreateTablet for tablet 1931e64fd5fc41cd9f4818dd179bc7a2 (DEFAULT_TABLE table=test-table [id=d510aee5e6a64ba5a35b036be0dd001d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:43:54.596676 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1931e64fd5fc41cd9f4818dd179bc7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:54.609854 9139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.565627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:33025 (local address 127.0.0.1:33376)
0805 22:43:54.567851 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:54.567897 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:54.567965 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:54.569670 (+ 1705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:54.569670 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:54.569672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:54.569725 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.569727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.576315 (+ 6588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.576316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.581241 (+ 4925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.581245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.609521 (+ 28276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.609523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.609610 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.609616 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 22:43:54.609686 (+ 70us) client_negotiation.cc:240] Negotiation successful
0805 22:43:54.609758 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2187,"thread_start_us":113,"threads_started":1}
I20260805 22:43:54.610277 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.565946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33376 (local address 127.23.197.67:33025)
0805 22:43:54.569240 (+ 3294us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:54.569245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:54.569254 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:43:54.569292 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:54.569294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:54.569310 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:54.574834 (+ 5524us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:54.574933 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.576193 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.586929 (+ 10736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.609290 (+ 22361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.609378 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.609469 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:54.610136 (+ 667us) server_negotiation.cc:299] Negotiation successful
0805 22:43:54.610189 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3251,"thread_start_us":109,"threads_started":1}
I20260805 22:43:54.611021 9019 tablet_service.cc:1467] Processing CreateTablet for tablet 1931e64fd5fc41cd9f4818dd179bc7a2 (DEFAULT_TABLE table=test-table [id=d510aee5e6a64ba5a35b036be0dd001d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:43:54.611425 9019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1931e64fd5fc41cd9f4818dd179bc7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:43:54.615413 9151 tablet_bootstrap.cc:492] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f: Bootstrap starting.
I20260805 22:43:54.615767 9149 tablet_bootstrap.cc:492] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: Bootstrap starting.
I20260805 22:43:54.616516 9151 tablet_bootstrap.cc:654] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:54.617000 9151 log.cc:824] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:54.617424 9149 tablet_bootstrap.cc:654] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:54.617825 9149 log.cc:824] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:54.623513 9152 tablet_bootstrap.cc:492] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: Bootstrap starting.
I20260805 22:43:54.624922 9152 tablet_bootstrap.cc:654] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: Neither blocks nor log segments found. Creating new log.
I20260805 22:43:54.625370 9152 log.cc:824] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: Log is configured to *not* fsync() on all Append() calls
I20260805 22:43:54.626322 9152 tablet_bootstrap.cc:492] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: No bootstrap required, opened a new log
I20260805 22:43:54.626423 9152 ts_tablet_manager.cc:1397] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260805 22:43:54.628750 9152 raft_consensus.cc:348] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } }
I20260805 22:43:54.629001 9152 raft_consensus.cc:374] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:54.629051 9152 raft_consensus.cc:729] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9403671dc4b4da292df571f3c9d35cd, State: Initialized, Role: FOLLOWER
I20260805 22:43:54.629684 9151 tablet_bootstrap.cc:492] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f: No bootstrap required, opened a new log
I20260805 22:43:54.629787 9151 ts_tablet_manager.cc:1397] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260805 22:43:54.629994 9149 tablet_bootstrap.cc:492] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: No bootstrap required, opened a new log
I20260805 22:43:54.630088 9149 ts_tablet_manager.cc:1397] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260805 22:43:54.632241 9149 raft_consensus.cc:348] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } }
I20260805 22:43:54.632463 9149 raft_consensus.cc:374] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:54.632514 9149 raft_consensus.cc:729] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9def1dc42ab4a3788db1f171bc93adc, State: Initialized, Role: FOLLOWER
I20260805 22:43:54.632670 9149 consensus_queue.cc:260] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } }
I20260805 22:43:54.632731 9152 consensus_queue.cc:260] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } }
I20260805 22:43:54.632997 9149 ts_tablet_manager.cc:1428] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:43:54.633153 8922 heartbeater.cc:503] Master 127.23.197.126:38413 was elected leader, sending a full tablet report...
I20260805 22:43:54.633981 9151 raft_consensus.cc:348] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } }
I20260805 22:43:54.634173 9151 raft_consensus.cc:374] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:43:54.634253 9151 raft_consensus.cc:729] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b82bb1bd3c4cf4b25a70f39aa3897f, State: Initialized, Role: FOLLOWER
I20260805 22:43:54.634400 9151 consensus_queue.cc:260] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } }
I20260805 22:43:54.634717 9151 ts_tablet_manager.cc:1428] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 22:43:54.634869 8715 heartbeater.cc:503] Master 127.23.197.126:38413 was elected leader, sending a full tablet report...
I20260805 22:43:54.634985 9152 ts_tablet_manager.cc:1428] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: Time spent starting tablet: real 0.009s user 0.001s sys 0.003s
I20260805 22:43:54.635134 9116 heartbeater.cc:503] Master 127.23.197.126:38413 was elected leader, sending a full tablet report...
W20260805 22:43:54.701025 8716 tablet.cc:2369] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:43:54.723090 8923 tablet.cc:2369] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 05 22:43:54 dist-test-slave-g170 krb5kdc[7663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
W20260805 22:43:54.771498 9120 tablet.cc:2369] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:43:54.834743 9159 raft_consensus.cc:482] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:43:54.834923 9159 raft_consensus.cc:504] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } }
I20260805 22:43:54.835340 9159 leader_election.cc:288] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8b82bb1bd3c4cf4b25a70f39aa3897f (127.23.197.65:40787), c9403671dc4b4da292df571f3c9d35cd (127.23.197.67:33025)
I20260805 22:43:54.840148 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.835984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:41027 (local address 127.23.197.65:40787)
0805 22:43:54.836035 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:54.836038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:54.836213 (+ 175us) server_negotiation.cc:407] Connection header received
0805 22:43:54.836339 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:54.836340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:54.836352 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:54.836383 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:54.836605 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.837491 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.839136 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.839749 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.839799 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.839809 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:54.840022 (+ 213us) server_negotiation.cc:299] Negotiation successful
0805 22:43:54.840063 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:43:54.840642 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.835606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:40787 (local address 127.23.197.66:41027)
0805 22:43:54.836132 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:54.836145 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:54.836261 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:54.836450 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:54.836451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:54.836452 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:54.836534 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.836537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.837636 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.837638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.839026 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.839030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.839872 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.839873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.839947 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.839950 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:43:54.840492 (+ 542us) client_negotiation.cc:240] Negotiation successful
0805 22:43:54.840545 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":69,"threads_started":1}
I20260805 22:43:54.841174 8629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1931e64fd5fc41cd9f4818dd179bc7a2" candidate_uuid: "c9def1dc42ab4a3788db1f171bc93adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" is_pre_election: true
I20260805 22:43:54.841361 8629 raft_consensus.cc:2454] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9def1dc42ab4a3788db1f171bc93adc in term 0.
I20260805 22:43:54.841639 8750 leader_election.cc:302] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8b82bb1bd3c4cf4b25a70f39aa3897f, c9def1dc42ab4a3788db1f171bc93adc; no voters:
I20260805 22:43:54.841784 9159 raft_consensus.cc:2781] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:43:54.841820 9159 raft_consensus.cc:482] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:43:54.841841 9159 raft_consensus.cc:3037] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:54.842926 9159 raft_consensus.cc:504] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } }
I20260805 22:43:54.843073 9159 leader_election.cc:288] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [CANDIDATE]: Term 1 election: Requested vote from peers c8b82bb1bd3c4cf4b25a70f39aa3897f (127.23.197.65:40787), c9403671dc4b4da292df571f3c9d35cd (127.23.197.67:33025)
I20260805 22:43:54.843777 8629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1931e64fd5fc41cd9f4818dd179bc7a2" candidate_uuid: "c9def1dc42ab4a3788db1f171bc93adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f"
I20260805 22:43:54.843858 8629 raft_consensus.cc:3037] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:54.844955 8629 raft_consensus.cc:2454] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9def1dc42ab4a3788db1f171bc93adc in term 1.
I20260805 22:43:54.845155 8750 leader_election.cc:302] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8b82bb1bd3c4cf4b25a70f39aa3897f, c9def1dc42ab4a3788db1f171bc93adc; no voters:
I20260805 22:43:54.845290 9159 raft_consensus.cc:2781] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:43:54.845466 9159 raft_consensus.cc:686] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 1 LEADER]: Becoming Leader. State: Replica: c9def1dc42ab4a3788db1f171bc93adc, State: Running, Role: LEADER
I20260805 22:43:54.845570 9159 consensus_queue.cc:237] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } }
I20260805 22:43:54.846551 8378 catalog_manager.cc:5495] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc reported cstate change: term changed from 0 to 1, leader changed from <none> to c9def1dc42ab4a3788db1f171bc93adc (127.23.197.66). New cstate: current_term: 1 leader_uuid: "c9def1dc42ab4a3788db1f171bc93adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9def1dc42ab4a3788db1f171bc93adc" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 38903 } health_report { overall_health: HEALTHY } } }
I20260805 22:43:54.850468 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.835887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:41777 (local address 127.23.197.67:33025)
0805 22:43:54.835956 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:54.835958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:54.846620 (+ 10662us) server_negotiation.cc:407] Connection header received
0805 22:43:54.846732 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:54.846733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:54.846747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:54.846778 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:43:54.847183 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.847934 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.849481 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.850050 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.850100 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.850119 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:54.850352 (+ 233us) server_negotiation.cc:299] Negotiation successful
0805 22:43:54.850393 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260805 22:43:54.850903 9170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.835813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:33025 (local address 127.23.197.66:41777)
0805 22:43:54.846533 (+ 10720us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:54.846547 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:43:54.846653 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:43:54.846846 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:43:54.846847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:43:54.846848 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:43:54.846928 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.846931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.848130 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.848132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.849366 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:43:54.849370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.850191 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.850192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:43:54.850272 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.850276 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:43:54.850756 (+ 480us) client_negotiation.cc:240] Negotiation successful
0805 22:43:54.850806 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10703,"thread_start_us":71,"threads_started":1}
I20260805 22:43:54.851466 9025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1931e64fd5fc41cd9f4818dd179bc7a2" candidate_uuid: "c9def1dc42ab4a3788db1f171bc93adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9403671dc4b4da292df571f3c9d35cd" is_pre_election: true
I20260805 22:43:54.851630 9025 raft_consensus.cc:2454] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9def1dc42ab4a3788db1f171bc93adc in term 0.
I20260805 22:43:54.851748 9042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1931e64fd5fc41cd9f4818dd179bc7a2" candidate_uuid: "c9def1dc42ab4a3788db1f171bc93adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9403671dc4b4da292df571f3c9d35cd"
I20260805 22:43:54.851800 9042 raft_consensus.cc:3037] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:43:54.852882 9042 raft_consensus.cc:2454] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9def1dc42ab4a3788db1f171bc93adc in term 1.
I20260805 22:43:54.861169 9148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.857361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.23.197.66:38903)
0805 22:43:54.857425 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0805 22:43:54.857427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:43:54.857439 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:43:54.857487 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:43:54.857488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:43:54.857515 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:43:54.857608 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:43:54.857872 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.858674 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.859901 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:43:54.860229 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:43:54.860291 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:43:54.860527 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:43:54.860671 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:43:54.860747 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 22:43:54.861020 (+ 273us) server_negotiation.cc:299] Negotiation successful
0805 22:43:54.861045 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260805 22:43:54.863618 9042 raft_consensus.cc:1261] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [term 1 FOLLOWER]: Refusing update from remote peer c9def1dc42ab4a3788db1f171bc93adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:43:54.864065 9171 consensus_queue.cc:1035] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9403671dc4b4da292df571f3c9d35cd" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:43:54.865787 8629 raft_consensus.cc:1261] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c8b82bb1bd3c4cf4b25a70f39aa3897f [term 1 FOLLOWER]: Refusing update from remote peer c9def1dc42ab4a3788db1f171bc93adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:43:54.866127 9171 consensus_queue.cc:1035] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8b82bb1bd3c4cf4b25a70f39aa3897f" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 40787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:43:54.868249 9174 mvcc.cc:205] Tried to move back new op lower bound from 7315332443598209024 to 7315332443528056832. Current Snapshot: MvccSnapshot[applied={T|T < 7315332443598209024}]
I20260805 22:43:54.868495 9175 mvcc.cc:205] Tried to move back new op lower bound from 7315332443598209024 to 7315332443528056832. Current Snapshot: MvccSnapshot[applied={T|T < 7315332443598209024}]
I20260805 22:43:54.871800 8378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:43:54.871912 8378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:43:54.873878 8378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7ce8496bf854736b79e3e6ece3a47e8: Sending DeleteTablet for 3 replicas of tablet 1931e64fd5fc41cd9f4818dd179bc7a2
I20260805 22:43:54.874317 8606 tablet_service.cc:1514] Processing DeleteTablet for tablet 1931e64fd5fc41cd9f4818dd179bc7a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:43:54 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:54386
I20260805 22:43:54.874985 9019 tablet_service.cc:1514] Processing DeleteTablet for tablet 1931e64fd5fc41cd9f4818dd179bc7a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:43:54 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:33376
I20260805 22:43:54.875677 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 8508
I20260805 22:43:54.879055 9185 tablet_replica.cc:330] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: stopping tablet replica
I20260805 22:43:54.879192 9185 raft_consensus.cc:2229] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:43:54.880940 9185 raft_consensus.cc:2258] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:43:54.885224 9185 ts_tablet_manager.cc:1905] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:43:54.886123 8818 tablet_service.cc:1514] Processing DeleteTablet for tablet 1931e64fd5fc41cd9f4818dd179bc7a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:43:54 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:56810
I20260805 22:43:54.887332 9185 ts_tablet_manager.cc:1918] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:43:54.887409 9185 log.cc:1196] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/1931e64fd5fc41cd9f4818dd179bc7a2
I20260805 22:43:54.887715 9185 ts_tablet_manager.cc:1939] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9403671dc4b4da292df571f3c9d35cd: Deleting consensus metadata
I20260805 22:43:54.888180 8353 catalog_manager.cc:4841] TS c9403671dc4b4da292df571f3c9d35cd (127.23.197.67:33025): tablet 1931e64fd5fc41cd9f4818dd179bc7a2 (table test-table [id=d510aee5e6a64ba5a35b036be0dd001d]) successfully deleted
I20260805 22:43:54.892882 9187 tablet_replica.cc:330] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: stopping tablet replica
I20260805 22:43:54.893117 9187 raft_consensus.cc:2229] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:43:54.893334 9187 raft_consensus.cc:2258] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:43:54.893851 9187 ts_tablet_manager.cc:1905] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:43:54.895589 9187 ts_tablet_manager.cc:1918] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:43:54.895722 9187 log.cc:1196] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/1931e64fd5fc41cd9f4818dd179bc7a2
I20260805 22:43:54.896063 9187 ts_tablet_manager.cc:1939] T 1931e64fd5fc41cd9f4818dd179bc7a2 P c9def1dc42ab4a3788db1f171bc93adc: Deleting consensus metadata
I20260805 22:43:54.896605 8352 catalog_manager.cc:4841] TS c9def1dc42ab4a3788db1f171bc93adc (127.23.197.66:38903): tablet 1931e64fd5fc41cd9f4818dd179bc7a2 (table test-table [id=d510aee5e6a64ba5a35b036be0dd001d]) successfully deleted
W20260805 22:43:54.899204 8352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:40787 (error 108)
I20260805 22:43:54.899426 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 8723
I20260805 22:43:54.910857 9139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:43:54.910679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:40787 (local address 127.0.0.1:54394)
0805 22:43:54.910755 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0805 22:43:54.910812 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:40787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260805 22:43:54.911163 8352 catalog_manager.cc:4584] TS c8b82bb1bd3c4cf4b25a70f39aa3897f (127.23.197.65:40787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1931e64fd5fc41cd9f4818dd179bc7a2: Network error: Client connection negotiation failed: client connection to 127.23.197.65:40787: connect: Connection refused (error 111)
I20260805 22:43:54.915071 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 8929
I20260805 22:43:54.928084 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 8328
I20260805 22:43:54.943097 24341 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260805 22:43:55.288362 24341 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260805 22:43:55.288491 24341 mini_ranger.cc:66] Stopping Ranger...
I20260805 22:43:55.634626 24341 mini_ranger.cc:68] Stopped Ranger
2026-08-05 22:43:55.634 UTC [7711] LOG: received smart shutdown request
2026-08-05 22:43:55.638 UTC [7711] LOG: background worker "logical replication launcher" (PID 7726) exited with exit code 1
2026-08-05 22:43:55.639 UTC [7716] LOG: shutting down
2026-08-05 22:43:55.813 UTC [7711] LOG: database system is shut down
2026-08-05T22:43:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49268 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:43:55 dist-test-slave-g170 krb5kdc[9200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:43:55 dist-test-slave-g170 krb5kdc[9200](info): set up 2 sockets
Aug 05 22:43:55 dist-test-slave-g170 krb5kdc[9200](info): commencing operation
krb5kdc: starting...
W20260805 22:43:57.216249 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:43:57 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:43:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.197.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.197.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/rangeradmin_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/rangeradmin_127.23.197.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.197.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.197.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/rangerlookup_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/rangerlookup_127.23.197.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.197.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.197.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/HTTP_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/HTTP_127.23.197.84@KRBTEST.COM.keytab.
I20260805 22:43:57.409807 24341 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskHuFw91/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/postgres -l logfile start
2026-08-05 22:44:00.075 UTC [9244] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-05 22:44:00.075 UTC [9244] LOG: listening on IPv4 address "127.23.197.84", port 48577
2026-08-05 22:44:00.079 UTC [9244] LOG: listening on Unix socket "/tmp/.s.PGSQL.48577"
2026-08-05 22:44:00.103 UTC [9250] LOG: database system was shut down at 2026-08-05 22:43:58 UTC
2026-08-05 22:44:00.108 UTC [9244] LOG: database system is ready to accept connections
I20260805 22:44:01.317991 24341 mini_postgres.cc:96] Postgres bound to 48577
2026-08-05 22:44:01.326 UTC [9259] FATAL: database "slave" does not exist
127.23.197.84:48577 - accepting connections
I20260805 22:44:01.327167 24341 mini_ranger.cc:161] Starting Ranger...
I20260805 22:44:01.343665 24341 mini_ranger.cc:84] Created miniranger Postgres user
I20260805 22:44:01.868690 24341 mini_ranger.cc:87] Created ranger Postgres database
I20260805 22:44:01.868820 24341 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-admin
2026-08-05 22:44:02,150 [I] DB FLAVOR :POSTGRES
2026-08-05 22:44:02,151 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 22:44:02,151 [I] Checking connection..
2026-08-05 22:44:02,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-05 22:44:02,403 [I] Checking connection passed.
2026-08-05 22:44:02,404 [I] --------- Verifying version history table ---------
2026-08-05 22:44:02,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 22:44:02,685 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 22:44:02,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 22:44:02,936 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 22:44:02,937 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-05 22:44:02,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-05 22:44:03.206 UTC [9361] WARNING: there is no transaction in progress
2026-08-05 22:44:03,216 [I] create_dbversion_catalog.sql file imported successfully
2026-08-05 22:44:03,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 22:44:03,464 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-05 22:44:03,464 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 22:44:03,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-05 22:44:03,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-05 22:44:04,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-g170.c.gcp-upstream.internal','N') ;"
2026-08-05 22:44:04,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 22:44:04,585 [I] Table x_portal_user does not exist in database ranger
2026-08-05 22:44:04,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 22:44:04,877 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-05 22:44:04,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 22:44:05,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 22:44:05,363 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-05 22:44:05,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-05 22:44:06.381 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.400 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.418 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.435 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.453 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.470 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.530 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.537 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.553 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.564 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.575 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.590 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.600 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.607 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.615 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:06.622 UTC [9530] WARNING: there is no transaction in progress
2026-08-05 22:44:07,324 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-05 22:44:07,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-05 22:44:07,558 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-05 22:44:07,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 22:44:07,828 [I] Table x_portal_user already exists in database 'ranger'
2026-08-05 22:44:07,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 22:44:08,103 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-05 22:44:08,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 22:44:08,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 22:44:08,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-g170.c.gcp-upstream.internal';"
2026-08-05 22:44:08,867 [I] CORE_DB_SCHEMA import status has been updated
2026-08-05 22:44:08,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-05 22:44:09,114 [I] DB_PATCHES have already been applied
I20260805 22:44:09.120285 24341 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-admin:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskHuFw91/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260805 22:44:09.120388 24341 mini_ranger.cc:193] Using host: 127.23.197.84
I20260805 22:44:09.124745 24341 mini_ranger.cc:227] Ranger admin URL: http://127.23.197.84:52027
Aug 05, 2026 10:44:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 10:44:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 10:44:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-admin/ews/webapp
Aug 05, 2026 10:44:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 05, 2026 10:44:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 05, 2026 10:44:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-admin/ews/webapp].
Aug 05, 2026 10:44:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260805 22:44:10.510835 24341 mini_ranger.cc:160] Time spent starting Ranger: real 9.184s user 0.002s sys 0.010s
Aug 05, 2026 10:44:32 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 05, 2026 10:44:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 05, 2026 10:44:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 05, 2026 10:44:32 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 05, 2026 10:44:32 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260805 22:44:33.589824 24341 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.197.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.197.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/rangerkms_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.197.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/rangerkms_127.23.197.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.197.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.197.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.197.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/HTTP_127.23.197.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.197.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/HTTP_127.23.197.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 05 22:44:33 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969873, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260805 22:44:33.680030 24341 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260805 22:44:33.691020 24341 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260805 22:44:34.633671 24341 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260805 22:44:34.633780 24341 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-kms
I20260805 22:44:34.633802 24341 mini_ranger_kms.cc:227] Using postgres at 127.23.197.84:48577
2026-08-05 22:44:34,835 [I] DB FLAVOR :POSTGRES
2026-08-05 22:44:34,835 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 22:44:34,835 [I] Checking connection
2026-08-05 22:44:34,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-05 22:44:35,130 [I] connection success
2026-08-05 22:44:35,130 [I] --------- Verifying Ranger DB tables ---------
2026-08-05 22:44:35,131 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-05 22:44:35,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-05 22:44:35,410 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-05 22:44:35,411 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 22:44:35,411 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-05 22:44:35,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.197.84:48577/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-05 22:44:35,704 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260805 22:44:36.226953 24341 mini_ranger_kms.cc:310] Created kms service
I20260805 22:44:36.447501 24341 mini_ranger_kms.cc:326] Created kudu user
I20260805 22:44:36.509725 24341 mini_ranger_kms.cc:343] Created rangerkms user
Aug 05 22:44:36 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969873, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:36.986042 24341 mini_ranger_kms.cc:384] Added ranger policy
I20260805 22:44:36.986302 24341 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-kms:/tmp/dist-test-taskHuFw91/build/release/bin/postgresql.jar:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskHuFw91/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260805 22:44:36.986332 24341 mini_ranger_kms.cc:240] Using host: 127.23.197.84
I20260805 22:44:36.990913 24341 mini_ranger_kms.cc:276] Ranger KMS PID: 9790
I20260805 22:44:36.991015 24341 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.197.84:35403
Aug 05, 2026 10:44:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 10:44:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 10:44:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 05, 2026 10:44:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 05, 2026 10:44:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-kms/ews/webapp].
Aug 05, 2026 10:44:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 05 22:44:39 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785969879, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.197.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:39.862635 24341 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.183s user 0.013s sys 0.172s
I20260805 22:44:39.862744 24341 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260805 22:44:39.862794 24341 mini_ranger_kms.cc:401] 127.23.197.84:35403/kms/v1/keys
Aug 05 22:44:40 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969879, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.197.84@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
Aug 05 22:44:40 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969879, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.197.84@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
Aug 05 22:44:42 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:44:42.760478 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40357
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:40357
--ranger_config_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:35403/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:42.855448 9864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:42.855656 9864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:42.855679 9864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:42.857061 9864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:44:42.857107 9864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:42.857122 9864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:42.857134 9864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:44:42.857146 9864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:44:42.858728 9864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.197.84:35403/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:40357
--ranger_config_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:42.858984 9864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:42.859243 9864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:42.864521 9871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:42.864521 9874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:42.864996 9864 server_base.cc:1034] running on GCE node
W20260805 22:44:42.864579 9872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:42.865288 9864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:42.865646 9864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:42.866804 9864 hybrid_clock.cc:648] HybridClock initialized: now 1785969882866784 us; error 39 us; skew 500 ppm
Aug 05 22:44:42 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:42.875228 9864 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:44:42.875782 9864 webserver.cc:460] Webserver started at http://127.23.197.126:45563/ using document root <none> and password file <none>
I20260805 22:44:42.876024 9864 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:42.876066 9864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:42.876209 9864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 22:44:42 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:42.939221 9864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "03462265d01a4fb1978873dc83c77030"
format_stamp: "Formatted at 2026-08-05 22:44:42 on dist-test-slave-g170"
server_key: "354d79e53dcfc23af929f70521d6e19a"
server_key_iv: "8703d0f8ea27553021cc0ab4018b1cca"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:42.939719 9864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "03462265d01a4fb1978873dc83c77030"
format_stamp: "Formatted at 2026-08-05 22:44:42 on dist-test-slave-g170"
server_key: "354d79e53dcfc23af929f70521d6e19a"
server_key_iv: "8703d0f8ea27553021cc0ab4018b1cca"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:42.941691 9864 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260805 22:44:42.958379 9885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:42.958678 9864 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:44:42.958807 9864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "03462265d01a4fb1978873dc83c77030"
format_stamp: "Formatted at 2026-08-05 22:44:42 on dist-test-slave-g170"
server_key: "354d79e53dcfc23af929f70521d6e19a"
server_key_iv: "8703d0f8ea27553021cc0ab4018b1cca"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:42.958915 9864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:42.973985 9864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:42.976423 9864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:42.976594 9864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:42.982380 9864 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:40357
I20260805 22:44:42.982460 9966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:40357 every 8 connection(s)
I20260805 22:44:42.982899 9864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:44:42.983897 9967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:42.986506 9967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Bootstrap starting.
I20260805 22:44:42.987164 9967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:42.987476 9967 log.cc:824] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:42.988343 9967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: No bootstrap required, opened a new log
I20260805 22:44:42.989888 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 9864
I20260805 22:44:42.990015 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:44:42.990226 24341 external_mini_cluster.cc:1428] Setting key 1f6753cf17e5e810d303dd2f0bfccbb0
I20260805 22:44:42.991026 9967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } }
I20260805 22:44:42.991241 9967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:42.991281 9967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03462265d01a4fb1978873dc83c77030, State: Initialized, Role: FOLLOWER
I20260805 22:44:42.991469 9967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } }
I20260805 22:44:42.991595 9967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:44:42.991669 9967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:44:42.991739 9967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:42.992378 9967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } }
I20260805 22:44:42.992496 9967 leader_election.cc:302] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03462265d01a4fb1978873dc83c77030; no voters:
I20260805 22:44:42.992707 9967 leader_election.cc:288] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:44:42.992789 9974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:44:42.993048 9974 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 1 LEADER]: Becoming Leader. State: Replica: 03462265d01a4fb1978873dc83c77030, State: Running, Role: LEADER
I20260805 22:44:42.993108 9967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:44:42.993209 9974 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } }
I20260805 22:44:42.993628 9974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03462265d01a4fb1978873dc83c77030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } } }
I20260805 22:44:42.993762 9974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:42.994009 9976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03462265d01a4fb1978873dc83c77030. Latest consensus state: current_term: 1 leader_uuid: "03462265d01a4fb1978873dc83c77030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } } }
I20260805 22:44:42.994161 9976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:42.994753 9967 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260805 22:44:42.998965 9988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:44:42.999760 9988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 05 22:44:42 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:43.005967 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:42.990835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57018 (local address 127.23.197.126:40357)
0805 22:44:42.991053 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:42.991064 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:42.991099 (+ 35us) server_negotiation.cc:407] Connection header received
0805 22:44:42.991223 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:42.991230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:42.991305 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:42.991465 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:42.992298 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:42.993837 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:42.994951 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:42.997868 (+ 2917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:42.998077 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:43.001218 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:43.001222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:43.001223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:43.001235 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:43.003730 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:43.004161 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:43.004165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:43.004165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:43.004290 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:43.004524 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:43.004525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:43.004526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:43.004897 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:43.005050 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:43.005519 (+ 469us) server_negotiation.cc:299] Negotiation successful
0805 22:44:43.005681 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":97,"threads_started":1}
I20260805 22:44:43.011658 9988 catalog_manager.cc:1347] Generated new cluster ID: 7094bc1af6e14912a8a0a13c0288f1ce
I20260805 22:44:43.011732 9988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:44:43.019258 9988 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:44:43.020058 9988 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:44:43.029558 9988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Generated new TSK 0
I20260805 22:44:43.030565 9988 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:44:43.856341 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:35403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40357
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:43.983896 10041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:43.984223 10041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:43.984288 10041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:43.986146 10041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:43.986215 10041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:43.986259 10041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:44:43.988535 10041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.197.84:35403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:43.988845 10041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:43.989133 10041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:43.995179 10053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:43.995179 10050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:43.995443 10051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:43.995945 10041 server_base.cc:1034] running on GCE node
I20260805 22:44:43.996198 10041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:43.996497 10041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:43.997709 10041 hybrid_clock.cc:648] HybridClock initialized: now 1785969883997692 us; error 42 us; skew 500 ppm
Aug 05 22:44:43 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969883, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:44.000872 10041 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:44:44.001463 10041 webserver.cc:460] Webserver started at http://127.23.197.65:34485/ using document root <none> and password file <none>
I20260805 22:44:44.001781 10041 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:44.001849 10041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:44.001981 10041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 22:44:43 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969883, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:44.017810 10041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "b3a7ff2a92454ff98d9079d920a2a88a"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "13501b21470ebf88bc603231a29dc429"
server_key_iv: "67df19edaccf9100585cb2671db88ade"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.018362 10041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "b3a7ff2a92454ff98d9079d920a2a88a"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "13501b21470ebf88bc603231a29dc429"
server_key_iv: "67df19edaccf9100585cb2671db88ade"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.020819 10041 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260805 22:44:44.039384 10066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.039738 10041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 22:44:44.039866 10041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "b3a7ff2a92454ff98d9079d920a2a88a"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "13501b21470ebf88bc603231a29dc429"
server_key_iv: "67df19edaccf9100585cb2671db88ade"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.039985 10041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:44.072178 10041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:44.074666 10041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:44.074864 10041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:44.075224 10041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:44.075706 10041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:44.075778 10041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.075865 10041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:44.075901 10041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.084321 10041 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:38969
I20260805 22:44:44.084398 10233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:38969 every 8 connection(s)
I20260805 22:44:44.084930 10041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:44:44.087009 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 10041
I20260805 22:44:44.087133 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:44:44.087358 24341 external_mini_cluster.cc:1428] Setting key 397a310b6d2495a2964a181b88b7ee03
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969883, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:44.098278 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:44.085909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40357 (local address 127.23.197.65:53771)
0805 22:44:44.086191 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:44.086243 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:44.086916 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:44.087272 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:44.087276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:44.087545 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:44.087831 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:44.087837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.088853 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.088854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:44.090262 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:44.090265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.090709 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.090710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:44.090856 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:44.091460 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:44.091472 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:44.093547 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:44.096216 (+ 2669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:44.096227 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:44.096230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:44.096596 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:44.096918 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:44.096919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:44.096920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:44.097069 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:44.097428 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:44.097436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:44.097635 (+ 199us) client_negotiation.cc:769] Sending connection context
0805 22:44:44.097778 (+ 143us) client_negotiation.cc:240] Negotiation successful
0805 22:44:44.097965 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":103,"threads_started":1}
I20260805 22:44:44.098765 10237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:44.085957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:53771 (local address 127.23.197.126:40357)
0805 22:44:44.086099 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:44.086101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:44.086354 (+ 253us) server_negotiation.cc:407] Connection header received
0805 22:44:44.087049 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:44.087050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:44.087067 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:44.087162 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:44.087951 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.088731 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.090364 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.090595 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.090686 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:44.093697 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:44.093699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:44.093700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:44.093710 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:44.096097 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:44.096713 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:44.096715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:44.096716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:44.096834 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:44.097157 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:44.097158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:44.097158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:44.097322 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:44.097394 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:44.098537 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0805 22:44:44.098628 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260805 22:44:44.099490 10234 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40357
I20260805 22:44:44.099601 10234 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:44.099813 10234 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
I20260805 22:44:44.100478 9920 ts_manager.cc:194] Registered new tserver with Master: b3a7ff2a92454ff98d9079d920a2a88a (127.23.197.65:38969)
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
I20260805 22:44:44.101472 9920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:53771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:44:44.128250 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:35403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40357
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785969884, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260805 22:44:44.260723 10243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:44.261176 10243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:44.261274 10243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:44.263712 10243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:44.263783 10243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:44.263830 10243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:44:44.266512 10243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.197.84:35403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:44.266821 10243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:44.267186 10243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:44.276774 10252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:44.276795 10257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:44.277611 10253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:44.280886 10243 server_base.cc:1034] running on GCE node
I20260805 22:44:44.281172 10243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:44.281490 10243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:44.282666 10243 hybrid_clock.cc:648] HybridClock initialized: now 1785969884282633 us; error 49 us; skew 500 ppm
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969884, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:44.286409 10243 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:44:44.287031 10243 webserver.cc:460] Webserver started at http://127.23.197.66:40991/ using document root <none> and password file <none>
I20260805 22:44:44.287426 10243 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:44.287495 10243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:44.287642 10243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969884, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:44.303560 10243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "efd80be5ebcb4664ba45c341352faf7e"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "a7cfc26aed93228e237975e6af0768e0"
server_key_iv: "08bf17dfc5fd46c5b62077f5bc132fc5"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.304062 10243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "efd80be5ebcb4664ba45c341352faf7e"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "a7cfc26aed93228e237975e6af0768e0"
server_key_iv: "08bf17dfc5fd46c5b62077f5bc132fc5"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.309234 10243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260805 22:44:44.323128 10269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.323577 10243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 22:44:44.323720 10243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "efd80be5ebcb4664ba45c341352faf7e"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "a7cfc26aed93228e237975e6af0768e0"
server_key_iv: "08bf17dfc5fd46c5b62077f5bc132fc5"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.323875 10243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:44.346858 10243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:44.349658 10243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:44.349839 10243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:44.350077 10243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:44.350498 10243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:44.350553 10243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.350597 10243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:44.350627 10243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.357806 10243 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:33207
I20260805 22:44:44.358467 10243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:44:44.368659 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 10243
I20260805 22:44:44.368820 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:44:44.369042 24341 external_mini_cluster.cc:1428] Setting key 8de5e840c7b908a409535fcc852d42ca
I20260805 22:44:44.369827 10447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:33207 every 8 connection(s)
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969884, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:44.391925 10237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:44.372248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:46221 (local address 127.23.197.126:40357)
0805 22:44:44.372300 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:44.372303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:44.375770 (+ 3467us) server_negotiation.cc:407] Connection header received
0805 22:44:44.376548 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:44.376550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:44.376576 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:44.376608 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:44.377522 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.378282 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.381527 (+ 3245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.381862 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.381909 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:44.388156 (+ 6247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:44.388158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:44.388158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:44.388167 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:44.390033 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:44.390547 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:44.390549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:44.390549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:44.390611 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:44.390935 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:44.390950 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:44.390951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:44.391103 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:44.391161 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:44.391716 (+ 555us) server_negotiation.cc:299] Negotiation successful
0805 22:44:44.391824 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260805 22:44:44.392779 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:44.370554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40357 (local address 127.23.197.66:46221)
0805 22:44:44.375580 (+ 5026us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:44.375647 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:44.376403 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:44.376848 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:44.376852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:44.377118 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:44.377405 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:44.377412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.378437 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.378437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:44.379712 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:44.379715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.382020 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.382021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:44.382174 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:44.386317 (+ 4143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:44.386329 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:44.388036 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:44.390133 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:44.390140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:44.390143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:44.390449 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:44.390682 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:44.390682 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:44.390683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:44.390830 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:44.391199 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:44.391206 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:44.391544 (+ 338us) client_negotiation.cc:769] Sending connection context
0805 22:44:44.392311 (+ 767us) client_negotiation.cc:240] Negotiation successful
0805 22:44:44.392493 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4963,"thread_start_us":109,"threads_started":1}
I20260805 22:44:44.393676 10448 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40357
I20260805 22:44:44.393797 10448 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:44.394007 10448 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
I20260805 22:44:44.394527 9920 ts_manager.cc:194] Registered new tserver with Master: efd80be5ebcb4664ba45c341352faf7e (127.23.197.66:33207)
I20260805 22:44:44.395087 9920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:46221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:44:44.420584 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:35403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40357
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:44.549824 10454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:44.550091 10454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:44.550134 10454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:44.551712 10454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:44.551769 10454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:44.551806 10454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:44:44.554049 10454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.197.84:35403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:40357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:44.554306 10454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:44.554610 10454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:44.561906 10465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:44.562192 10463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:44.562577 10462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:44.563726 10454 server_base.cc:1034] running on GCE node
I20260805 22:44:44.563962 10454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:44.564255 10454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:44.565441 10454 hybrid_clock.cc:648] HybridClock initialized: now 1785969884565407 us; error 47 us; skew 500 ppm
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969884, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:44.568878 10454 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:44:44.569448 10454 webserver.cc:460] Webserver started at http://127.23.197.67:46787/ using document root <none> and password file <none>
I20260805 22:44:44.569763 10454 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:44.569828 10454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:44.569967 10454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969884, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:44.583235 10454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "8161d0c78c1f44e69094ed5ff2ced1eb"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "5e1c76473ec00b65d19bda395254d089"
server_key_iv: "d536fce7ca886c48615e5daa5d5bba4f"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.583703 10454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "8161d0c78c1f44e69094ed5ff2ced1eb"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "5e1c76473ec00b65d19bda395254d089"
server_key_iv: "d536fce7ca886c48615e5daa5d5bba4f"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.585387 10454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:44:44.599632 10475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.599892 10454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 22:44:44.600016 10454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "8161d0c78c1f44e69094ed5ff2ced1eb"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "5e1c76473ec00b65d19bda395254d089"
server_key_iv: "d536fce7ca886c48615e5daa5d5bba4f"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.600153 10454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:44.621768 10454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:44.624277 10454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:44.624451 10454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:44.624817 10454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:44.625255 10454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:44.625317 10454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.625361 10454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:44.625388 10454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.634904 10454 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:46455
I20260805 22:44:44.635346 10454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:44:44.636106 10646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:46455 every 8 connection(s)
I20260805 22:44:44.640259 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 10454
I20260805 22:44:44.640405 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:44:44.640610 24341 external_mini_cluster.cc:1428] Setting key 74365c6d14ea214ffbb1f013787efaa3
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969884, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:44.657038 10237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:44.636618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:49135 (local address 127.23.197.126:40357)
0805 22:44:44.636730 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:44.636732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:44.636745 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:44:44.637422 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:44.637424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:44.637439 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:44.637512 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:44.644331 (+ 6819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.645147 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.649633 (+ 4486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.649862 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.649972 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:44.653209 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:44.653211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:44.653211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:44.653220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:44.655126 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:44.655750 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:44.655752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:44.655752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:44.655829 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:44.656150 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:44.656151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:44.656152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:44.656313 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:44.656376 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:44.656827 (+ 451us) server_negotiation.cc:299] Negotiation successful
0805 22:44:44.656924 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260805 22:44:44.657923 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:44.636235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40357 (local address 127.23.197.67:49135)
0805 22:44:44.636423 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:44.636470 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:44.637244 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:44.637610 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:44.637614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:44.637897 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:44.638189 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:44.638196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.648110 (+ 9914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.648112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:44.649502 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:44.649507 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:44.650003 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:44.650004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:44.650167 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:44.650863 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:44.650875 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:44.653109 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:44.655236 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:44.655244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:44.655247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:44.655568 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:44.655909 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:44.655909 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:44.655910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:44.656055 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:44.656414 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:44.656421 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:44.656616 (+ 195us) client_negotiation.cc:769] Sending connection context
0805 22:44:44.657454 (+ 838us) client_negotiation.cc:240] Negotiation successful
0805 22:44:44.657658 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260805 22:44:44.658681 10647 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40357
I20260805 22:44:44.658787 10647 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:44.659003 10647 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
I20260805 22:44:44.659443 9920 ts_manager.cc:194] Registered new tserver with Master: 8161d0c78c1f44e69094ed5ff2ced1eb (127.23.197.67:46455)
I20260805 22:44:44.659951 9920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:49135
I20260805 22:44:44.666221 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:44:44.667214 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 10041
I20260805 22:44:44.713697 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 10243
I20260805 22:44:44.725098 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 10454
I20260805 22:44:44.734622 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 9864
I20260805 22:44:44.749456 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40357
--webserver_interface=127.23.197.126
--webserver_port=45563
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:40357
--ranger_config_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:35403/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.197.84:35403/kms with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:44.840858 10650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:44.841071 10650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:44.841094 10650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:44.842275 10650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:44:44.842319 10650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:44.842334 10650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:44.842346 10650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:44:44.842358 10650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:44:44.844071 10650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.197.84:35403/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:40357
--ranger_config_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=45563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:44.844326 10650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:44.844570 10650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:44.849647 10657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:44.849690 10656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:44.849647 10659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:44.849929 10650 server_base.cc:1034] running on GCE node
I20260805 22:44:44.850423 10650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:44.850755 10650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:44.851935 10650 hybrid_clock.cc:648] HybridClock initialized: now 1785969884851911 us; error 40 us; skew 500 ppm
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969884, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:44.855017 10650 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:44:44.855597 10650 webserver.cc:460] Webserver started at http://127.23.197.126:45563/ using document root <none> and password file <none>
I20260805 22:44:44.855865 10650 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:44.855940 10650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 22:44:44 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969884, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:44.938823 10650 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:44.939642 10670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:44.939812 10650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:44.939911 10650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "03462265d01a4fb1978873dc83c77030"
format_stamp: "Formatted at 2026-08-05 22:44:42 on dist-test-slave-g170"
server_key: "354d79e53dcfc23af929f70521d6e19a"
server_key_iv: "8703d0f8ea27553021cc0ab4018b1cca"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:44.940286 10650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:44.948602 10650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:44.950760 10650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:44.950970 10650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:44.955709 10650 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:40357
I20260805 22:44:44.955749 10747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:40357 every 8 connection(s)
I20260805 22:44:44.956149 10650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:44:44.957501 10748 sys_catalog.cc:263] Verifying existing consensus state
I20260805 22:44:44.957652 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 10650
I20260805 22:44:44.957980 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:35403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:38969
--local_ip_for_outbound_sockets=127.23.197.65
--tserver_master_addrs=127.23.197.126:40357
--webserver_port=34485
--webserver_interface=127.23.197.65
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.197.84:35403/kms with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:44:44.958384 10748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Bootstrap starting.
I20260805 22:44:44.966611 10748 log.cc:824] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:44.968554 10748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:44:44.968964 10748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Bootstrap complete.
I20260805 22:44:44.971390 10748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } }
I20260805 22:44:44.971666 10748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03462265d01a4fb1978873dc83c77030, State: Initialized, Role: FOLLOWER
I20260805 22:44:44.971899 10748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } }
I20260805 22:44:44.972002 10748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:44:44.972059 10748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:44:44.972107 10748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 1 FOLLOWER]: Advancing to term 2
I20260805 22:44:44.973274 10748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } }
I20260805 22:44:44.973407 10748 leader_election.cc:302] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03462265d01a4fb1978873dc83c77030; no voters:
I20260805 22:44:44.973623 10748 leader_election.cc:288] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 22:44:44.973712 10752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 2 FOLLOWER]: Leader election won for term 2
I20260805 22:44:44.973917 10752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [term 2 LEADER]: Becoming Leader. State: Replica: 03462265d01a4fb1978873dc83c77030, State: Running, Role: LEADER
I20260805 22:44:44.973927 10748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:44:44.974064 10752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } }
I20260805 22:44:44.975117 10753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "03462265d01a4fb1978873dc83c77030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } } }
I20260805 22:44:44.975201 10753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:44.975478 10773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:44:44.975793 10748 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260805 22:44:44.975991 10773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:44:44.976115 10773 catalog_manager.cc:1259] Loaded cluster ID: 7094bc1af6e14912a8a0a13c0288f1ce
I20260805 22:44:44.976148 10773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:44:44.976172 10755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03462265d01a4fb1978873dc83c77030. Latest consensus state: current_term: 2 leader_uuid: "03462265d01a4fb1978873dc83c77030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03462265d01a4fb1978873dc83c77030" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40357 } } }
I20260805 22:44:44.976250 10755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030 [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:44.980504 10773 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:44:44.981148 10773 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Loaded TSK: 0
I20260805 22:44:44.981467 10773 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 22:44:45.074997 10750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:45.075423 10750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:45.075547 10750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:45.077415 10750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:45.077636 10750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:45.077793 10750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:44:45.080031 10750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.197.84:35403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:38969
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=34485
--tserver_master_addrs=127.23.197.126:40357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:45.080503 10750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:45.080864 10750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:45.087231 10806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:45.087286 10803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:45.087839 10750 server_base.cc:1034] running on GCE node
W20260805 22:44:45.088896 10802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:45.089180 10750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:45.089470 10750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:45.090672 10750 hybrid_clock.cc:648] HybridClock initialized: now 1785969885090640 us; error 46 us; skew 500 ppm
Aug 05 22:44:45 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:45.094039 10750 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:44:45.094655 10750 webserver.cc:460] Webserver started at http://127.23.197.65:34485/ using document root <none> and password file <none>
I20260805 22:44:45.098038 10750 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:45.098201 10750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 22:44:45 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:45.113986 10750 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:45.115267 10819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:45.115509 10750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:44:45.115608 10750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "b3a7ff2a92454ff98d9079d920a2a88a"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "13501b21470ebf88bc603231a29dc429"
server_key_iv: "67df19edaccf9100585cb2671db88ade"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:45.116093 10750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:45.133899 10750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:45.136670 10750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:45.136987 10750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:45.137326 10750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:45.137773 10750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:45.137871 10750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:45.137977 10750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:45.138057 10750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:45.145354 10750 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:38969
I20260805 22:44:45.145938 10750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:44:45.149175 10974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:38969 every 8 connection(s)
I20260805 22:44:45.155181 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 10750
I20260805 22:44:45.155555 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:35403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:33207
--local_ip_for_outbound_sockets=127.23.197.66
--tserver_master_addrs=127.23.197.126:40357
--webserver_port=40991
--webserver_interface=127.23.197.66
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.197.84:35403/kms with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 22:44:45 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:45.162509 10981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:45.148326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40357 (local address 127.23.197.65:47379)
0805 22:44:45.149444 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:45.149487 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:45.150133 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:45.150896 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:45.150900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:45.151148 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:45.151421 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:45.151427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.153166 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.153168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:45.154443 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:45.154446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.155464 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.155465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:45.155592 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:45.156518 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:45.156529 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:45.158287 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:45.160618 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:45.160625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:45.160627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:45.160969 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:45.161253 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:45.161253 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:45.161254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:45.161372 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:45.161880 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:45.161885 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:45.162028 (+ 143us) client_negotiation.cc:769] Sending connection context
0805 22:44:45.162116 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 22:44:45.162274 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"thread_start_us":97,"threads_started":1}
I20260805 22:44:45.163130 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:45.148715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:47379 (local address 127.23.197.126:40357)
0805 22:44:45.150359 (+ 1644us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:45.150369 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:45.150405 (+ 36us) server_negotiation.cc:407] Connection header received
0805 22:44:45.150545 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:45.150551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:45.150605 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:45.150746 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:45.151559 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.153036 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.154546 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.155148 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.155344 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:45.158480 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:45.158485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:45.158485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:45.158498 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:45.160523 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:45.161060 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:45.161063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:45.161063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:45.161174 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:45.161435 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:45.161436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:45.161437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:45.161773 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:45.161851 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:45.162797 (+ 946us) server_negotiation.cc:299] Negotiation successful
0805 22:44:45.162916 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1570,"thread_start_us":128,"threads_started":1}
I20260805 22:44:45.163753 10978 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40357
I20260805 22:44:45.163851 10978 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:45.164050 10978 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
W20260805 22:44:45.164837 10978 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:40357 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260805 22:44:45.164908 10978 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:45.164979 10978 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
I20260805 22:44:45.165279 10978 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:45.165352 10978 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
W20260805 22:44:45.165592 10978 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 22:44:45.273514 10983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:45.273799 10983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:45.273836 10983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:45.275717 10983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:45.275784 10983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:45.275823 10983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:44:45.278091 10983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.197.84:35403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:33207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=40991
--tserver_master_addrs=127.23.197.126:40357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:45.278380 10983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:45.278653 10983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:45.288793 10988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:45.288897 10989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:45.289567 10983 server_base.cc:1034] running on GCE node
W20260805 22:44:45.289569 10991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:45.289891 10983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:45.290187 10983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:45.291393 10983 hybrid_clock.cc:648] HybridClock initialized: now 1785969885291369 us; error 44 us; skew 500 ppm
Aug 05 22:44:45 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:45.294376 10983 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:44:45.294978 10983 webserver.cc:460] Webserver started at http://127.23.197.66:40991/ using document root <none> and password file <none>
I20260805 22:44:45.295280 10983 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:45.295372 10983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 22:44:45 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:45.311266 10983 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:45.312341 11003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:45.312606 10983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:45.312724 10983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "efd80be5ebcb4664ba45c341352faf7e"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "a7cfc26aed93228e237975e6af0768e0"
server_key_iv: "08bf17dfc5fd46c5b62077f5bc132fc5"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:45.313251 10983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:45.336050 10983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:45.340353 10983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:45.340531 10983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:45.340850 10983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:45.341270 10983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:45.341342 10983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:45.341377 10983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:45.341394 10983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:45.349062 10983 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:33207
I20260805 22:44:45.349140 11164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:33207 every 8 connection(s)
I20260805 22:44:45.349582 10983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
Aug 05 22:44:45 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:45.359395 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 10983
I20260805 22:44:45.359776 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.197.84:35403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:46455
--local_ip_for_outbound_sockets=127.23.197.67
--tserver_master_addrs=127.23.197.126:40357
--webserver_port=46787
--webserver_interface=127.23.197.67
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.197.84:35403/kms with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:44:45.361596 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:45.351226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:50763 (local address 127.23.197.126:40357)
0805 22:44:45.351285 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:45.351288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:45.351299 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:44:45.351734 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:45.351735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:45.351747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:45.351809 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:45.352699 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.353281 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.354876 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.355082 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.355146 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:45.357753 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:45.357754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:45.357754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:45.357762 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:45.359658 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:45.360188 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:45.360189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:45.360190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:45.360254 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:45.360613 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:45.360614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:45.360615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:45.360819 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:45.360903 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:45.361331 (+ 428us) server_negotiation.cc:299] Negotiation successful
0805 22:44:45.361438 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260805 22:44:45.362228 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:45.350556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40357 (local address 127.23.197.66:50763)
0805 22:44:45.350852 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:45.350915 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:45.351609 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:45.351890 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:45.351895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:45.352152 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:45.352455 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:45.352462 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.353400 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.353401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:45.354764 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:45.354767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.355203 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.355205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:45.355359 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:45.355921 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:45.355934 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:45.357593 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:45.359757 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:45.359764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:45.359766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:45.360087 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:45.360376 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:45.360377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:45.360377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:45.360520 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:45.360937 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:45.360944 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:45.361134 (+ 190us) client_negotiation.cc:769] Sending connection context
0805 22:44:45.361704 (+ 570us) client_negotiation.cc:240] Negotiation successful
0805 22:44:45.361911 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":88,"threads_started":1}
I20260805 22:44:45.362780 11167 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40357
I20260805 22:44:45.362903 11167 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:45.363130 11167 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
W20260805 22:44:45.363827 11167 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:40357 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260805 22:44:45.363914 11167 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:45.363996 11167 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
I20260805 22:44:45.364360 11167 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:45.364450 11167 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
W20260805 22:44:45.364796 11167 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 22:44:45.478489 11171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:45.478742 11171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:45.478792 11171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:45.480147 11171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:45.480228 11171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:45.480278 11171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:44:45.482110 11171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.197.84:35403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:46455
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=46787
--tserver_master_addrs=127.23.197.126:40357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:45.482402 11171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:45.482702 11171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 22:44:45.491094 11171 server_base.cc:1034] running on GCE node
W20260805 22:44:45.491191 11181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:45.491436 11180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:45.491262 11186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:45.492897 11171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:45.493213 11171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:45.500597 11171 hybrid_clock.cc:648] HybridClock initialized: now 1785969885500569 us; error 48 us; skew 500 ppm
Aug 05 22:44:45 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:45.503746 11171 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:44:45.504348 11171 webserver.cc:460] Webserver started at http://127.23.197.67:46787/ using document root <none> and password file <none>
I20260805 22:44:45.504707 11171 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:45.504838 11171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 22:44:45 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:45.527734 11171 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260805 22:44:45.529299 11197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:45.529592 11171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:45.529768 11171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "8161d0c78c1f44e69094ed5ff2ced1eb"
format_stamp: "Formatted at 2026-08-05 22:44:44 on dist-test-slave-g170"
server_key: "5e1c76473ec00b65d19bda395254d089"
server_key_iv: "d536fce7ca886c48615e5daa5d5bba4f"
server_key_version: "kuduclusterkey@0"
I20260805 22:44:45.530230 11171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:45.548625 11171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:45.551265 11171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:45.551471 11171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:45.551734 11171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:45.552119 11171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:45.552178 11171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:45.552211 11171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:45.552241 11171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:45.559966 11171 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:46455
I20260805 22:44:45.560103 11384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:46455 every 8 connection(s)
I20260805 22:44:45.560469 11171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
Aug 05 22:44:45 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:45.571456 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:45.561349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:54099 (local address 127.23.197.126:40357)
0805 22:44:45.561408 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:45.561410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:45.561699 (+ 289us) server_negotiation.cc:407] Connection header received
0805 22:44:45.562377 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:45.562379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:45.562395 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:45.562468 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:45.563091 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.563606 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.564666 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.565008 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.565109 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:45.567808 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:45.567810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:45.567810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:45.567819 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:45.569727 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:45.570296 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:45.570297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:45.570298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:45.570367 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:45.570660 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:45.570661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:45.570661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:45.570818 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:45.570863 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:45.571247 (+ 384us) server_negotiation.cc:299] Negotiation successful
0805 22:44:45.571343 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260805 22:44:45.572129 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 11171
I20260805 22:44:45.573096 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:45.561345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40357 (local address 127.23.197.67:54099)
0805 22:44:45.561529 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:45.561590 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:45.562238 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:45.562557 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:45.562561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:45.562761 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:45.562965 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:45.562970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.563715 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.563716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:45.564542 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:45.564544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.565163 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.565164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:45.565349 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:45.566015 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:45.566031 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:45.567671 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:45.569838 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:45.569847 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:45.569850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:45.570183 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:45.570442 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:45.570443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:45.570443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:45.570585 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:45.570912 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:45.570918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:45.571091 (+ 173us) client_negotiation.cc:769] Sending connection context
0805 22:44:45.571883 (+ 792us) client_negotiation.cc:240] Negotiation successful
0805 22:44:45.572084 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260805 22:44:45.573796 11387 heartbeater.cc:347] Connected to a master server at 127.23.197.126:40357
I20260805 22:44:45.573904 11387 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:45.574116 11387 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
W20260805 22:44:45.574864 11387 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:40357 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260805 22:44:45.574945 11387 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:45.575024 11387 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
I20260805 22:44:45.575853 11387 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:45.575933 11387 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
W20260805 22:44:45.576262 11387 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 22:44:45.581694 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:45.572786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57026 (local address 127.23.197.126:40357)
0805 22:44:45.572826 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:45.572828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:45.572837 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:44:45.572916 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:45.572916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:45.572927 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:45.572981 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:45.574765 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.575450 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.576586 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:45.576893 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:45.576967 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:45.578479 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:45.578480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:45.578480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:45.578487 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:45.580355 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:45.580856 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:45.580857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:45.580858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:45.580937 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:45.581163 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:45.581169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:45.581169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:45.581295 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:45.581419 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:45.581551 (+ 132us) server_negotiation.cc:299] Negotiation successful
0805 22:44:45.581622 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 05 22:44:46 dist-test-slave-g170 krb5kdc[9200](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785969886, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:46.165787 10978 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:46.165979 10978 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
I20260805 22:44:46.166388 10692 ts_manager.cc:194] Registered new tserver with Master: b3a7ff2a92454ff98d9079d920a2a88a (127.23.197.65:38969)
I20260805 22:44:46.167142 10692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:47379
I20260805 22:44:46.364977 11167 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:46.365145 11167 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
I20260805 22:44:46.365476 10692 ts_manager.cc:194] Registered new tserver with Master: efd80be5ebcb4664ba45c341352faf7e (127.23.197.66:33207)
I20260805 22:44:46.366011 10692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:50763
Aug 05 22:44:46 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969886, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
I20260805 22:44:46.576464 11387 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:46.576644 11387 heartbeater.cc:511] Master 127.23.197.126:40357 requested a full tablet report, sending...
I20260805 22:44:46.577039 10692 ts_manager.cc:194] Registered new tserver with Master: 8161d0c78c1f44e69094ed5ff2ced1eb (127.23.197.67:46455)
I20260805 22:44:46.577469 10692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:54099
I20260805 22:44:46.578863 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:44:46.586552 11419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.579829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46884 (local address 127.23.197.126:40357)
0805 22:44:46.579958 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:46.579960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:46.580089 (+ 129us) server_negotiation.cc:407] Connection header received
0805 22:44:46.580164 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:46.580165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:46.580178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:46.580243 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:46.580878 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.581364 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.582095 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.582324 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.582423 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.583682 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:46.583683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:46.583684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:46.583692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:46.585300 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:46.585692 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:46.585694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:46.585695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:46.585758 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:46.586004 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:46.586005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:46.586005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:46.586119 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:46.586230 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:46.586334 (+ 104us) server_negotiation.cc:299] Negotiation successful
0805 22:44:46.586420 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260805 22:44:46.588613 10692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:44:46.590867 10692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:44:46.601130 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.593826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:33207 (local address 127.0.0.1:48082)
0805 22:44:46.596851 (+ 3025us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:46.596894 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:46.596966 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:46.597218 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:46.597221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:46.597224 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:46.597292 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.597294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.598579 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.598581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.599806 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.599809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.600812 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.600813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.600891 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.600906 (+ 15us) client_negotiation.cc:769] Sending connection context
0805 22:44:46.600990 (+ 84us) client_negotiation.cc:240] Negotiation successful
0805 22:44:46.601047 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2990,"thread_start_us":108,"threads_started":1}
I20260805 22:44:46.601213 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.594259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.23.197.66:33207)
0805 22:44:46.594381 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:46.594385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:46.597053 (+ 2668us) server_negotiation.cc:407] Connection header received
0805 22:44:46.597094 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:46.597097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:46.597114 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:46.597188 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:46.597374 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.598483 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.599901 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.600671 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.600748 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.600828 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:46.601015 (+ 187us) server_negotiation.cc:299] Negotiation successful
0805 22:44:46.601056 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
I20260805 22:44:46.601929 11044 tablet_service.cc:1467] Processing CreateTablet for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8 (DEFAULT_TABLE table=test-table [id=41398f624f614ca5a2c58e17f9c71f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:44:46.602353 11044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:46.606961 11434 tablet_bootstrap.cc:492] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: Bootstrap starting.
I20260805 22:44:46.607681 11434 tablet_bootstrap.cc:654] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:46.608007 11434 log.cc:824] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:46.608985 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.593828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:46455 (local address 127.0.0.1:44468)
0805 22:44:46.601337 (+ 7509us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:46.601347 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:46.601379 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:46.604050 (+ 2671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:46.604051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:46.604053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:46.604089 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.604090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.605977 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.605978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.607384 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.607388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.608666 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.608667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.608731 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.608735 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:44:46.608845 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 22:44:46.608893 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7497,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260805 22:44:46.609009 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.594120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44468 (local address 127.23.197.67:46455)
0805 22:44:46.603862 (+ 9742us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:46.603868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:46.603881 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:44:46.603917 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:46.603919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:46.603938 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:46.604041 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:46.604182 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.605843 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.607509 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.608554 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.608654 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.608718 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:46.608867 (+ 149us) server_negotiation.cc:299] Negotiation successful
0805 22:44:46.608914 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9700,"thread_start_us":90,"threads_started":1}
I20260805 22:44:46.610070 11249 tablet_service.cc:1467] Processing CreateTablet for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8 (DEFAULT_TABLE table=test-table [id=41398f624f614ca5a2c58e17f9c71f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:44:46.610448 11249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:46.612437 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.594111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51638 (local address 127.23.197.65:38969)
0805 22:44:46.594258 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:46.594263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:46.606581 (+ 12318us) server_negotiation.cc:407] Connection header received
0805 22:44:46.606641 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:46.606643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:46.606660 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:46.606746 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:46.608968 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.609878 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.611200 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.611922 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.611983 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.612035 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:46.612311 (+ 276us) server_negotiation.cc:299] Negotiation successful
0805 22:44:46.612360 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260805 22:44:46.612703 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.593833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:38969 (local address 127.0.0.1:51638)
0805 22:44:46.604184 (+ 10351us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:46.604196 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:46.604252 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:46.608823 (+ 4571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:46.608825 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:46.608828 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:46.608870 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.608872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.610016 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.610017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.611079 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.611082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.612111 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.612112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.612171 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.612175 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:44:46.612564 (+ 389us) client_negotiation.cc:240] Negotiation successful
0805 22:44:46.612613 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10331,"thread_start_us":107,"threads_started":1}
I20260805 22:44:46.613473 10877 tablet_service.cc:1467] Processing CreateTablet for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8 (DEFAULT_TABLE table=test-table [id=41398f624f614ca5a2c58e17f9c71f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:44:46.613848 10877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:46.614274 11436 tablet_bootstrap.cc:492] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: Bootstrap starting.
I20260805 22:44:46.614899 11436 tablet_bootstrap.cc:654] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:46.615191 11436 log.cc:824] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:46.616104 11436 tablet_bootstrap.cc:492] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: No bootstrap required, opened a new log
I20260805 22:44:46.616164 11436 ts_tablet_manager.cc:1397] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:44:46.618474 11438 tablet_bootstrap.cc:492] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a: Bootstrap starting.
I20260805 22:44:46.619319 11438 tablet_bootstrap.cc:654] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:46.619750 11438 log.cc:824] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:46.619901 11436 raft_consensus.cc:348] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } }
I20260805 22:44:46.620082 11436 raft_consensus.cc:374] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:46.620127 11436 raft_consensus.cc:729] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8161d0c78c1f44e69094ed5ff2ced1eb, State: Initialized, Role: FOLLOWER
I20260805 22:44:46.620306 11436 consensus_queue.cc:260] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } }
I20260805 22:44:46.620836 11434 tablet_bootstrap.cc:492] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: No bootstrap required, opened a new log
I20260805 22:44:46.620905 11434 ts_tablet_manager.cc:1397] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260805 22:44:46.621008 11438 tablet_bootstrap.cc:492] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a: No bootstrap required, opened a new log
I20260805 22:44:46.621088 11438 ts_tablet_manager.cc:1397] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 22:44:46.623234 11434 raft_consensus.cc:348] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } }
I20260805 22:44:46.623399 11434 raft_consensus.cc:374] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:46.623433 11434 raft_consensus.cc:729] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd80be5ebcb4664ba45c341352faf7e, State: Initialized, Role: FOLLOWER
I20260805 22:44:46.623602 11434 consensus_queue.cc:260] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } }
I20260805 22:44:46.623564 11438 raft_consensus.cc:348] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } }
I20260805 22:44:46.623730 11438 raft_consensus.cc:374] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:46.623767 11438 raft_consensus.cc:729] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a7ff2a92454ff98d9079d920a2a88a, State: Initialized, Role: FOLLOWER
I20260805 22:44:46.623880 11434 ts_tablet_manager.cc:1428] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:44:46.623996 11167 heartbeater.cc:503] Master 127.23.197.126:40357 was elected leader, sending a full tablet report...
I20260805 22:44:46.624166 11438 consensus_queue.cc:260] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } }
I20260805 22:44:46.624962 11438 ts_tablet_manager.cc:1428] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260805 22:44:46.625089 11436 ts_tablet_manager.cc:1428] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260805 22:44:46.625160 10978 heartbeater.cc:503] Master 127.23.197.126:40357 was elected leader, sending a full tablet report...
I20260805 22:44:46.625240 11387 heartbeater.cc:503] Master 127.23.197.126:40357 was elected leader, sending a full tablet report...
W20260805 22:44:46.650204 10979 tablet.cc:2369] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 05 22:44:46 dist-test-slave-g170 krb5kdc[9200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969886, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for HTTP/127.23.197.84@KRBTEST.COM
W20260805 22:44:46.812889 11388 tablet.cc:2369] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:44:46.815490 11441 raft_consensus.cc:482] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:44:46.815637 11441 raft_consensus.cc:504] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } }
I20260805 22:44:46.816083 11441 leader_election.cc:288] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efd80be5ebcb4664ba45c341352faf7e (127.23.197.66:33207), 8161d0c78c1f44e69094ed5ff2ced1eb (127.23.197.67:46455)
I20260805 22:44:46.820138 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.816367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:46425 (local address 127.23.197.67:46455)
0805 22:44:46.816419 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:46.816422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:46.816595 (+ 173us) server_negotiation.cc:407] Connection header received
0805 22:44:46.816713 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:46.816714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:46.816725 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:46.816783 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:46.817047 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.817761 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.819218 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.819751 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.819828 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.819837 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:46.820025 (+ 188us) server_negotiation.cc:299] Negotiation successful
0805 22:44:46.820063 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260805 22:44:46.820154 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.816272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:46455 (local address 127.23.197.65:46425)
0805 22:44:46.816525 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:46.816537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:46.816629 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:46.816865 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:46.816866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:46.816869 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:46.816961 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.816963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.817851 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.817852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.819133 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.819137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.819864 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.819866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.819938 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.819942 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:44:46.820011 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 22:44:46.820050 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":107,"threads_started":1}
I20260805 22:44:46.820510 11299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69e3bc3b1dbe48d8bb6c2fb884b3dec8" candidate_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" is_pre_election: true
I20260805 22:44:46.820660 11299 raft_consensus.cc:2454] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3a7ff2a92454ff98d9079d920a2a88a in term 0.
I20260805 22:44:46.820910 10821 leader_election.cc:302] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8161d0c78c1f44e69094ed5ff2ced1eb, b3a7ff2a92454ff98d9079d920a2a88a; no voters:
I20260805 22:44:46.821041 11441 raft_consensus.cc:2781] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:44:46.821087 11441 raft_consensus.cc:482] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:44:46.821125 11441 raft_consensus.cc:3037] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:46.822039 11441 raft_consensus.cc:504] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } }
I20260805 22:44:46.822201 11441 leader_election.cc:288] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [CANDIDATE]: Term 1 election: Requested vote from peers efd80be5ebcb4664ba45c341352faf7e (127.23.197.66:33207), 8161d0c78c1f44e69094ed5ff2ced1eb (127.23.197.67:46455)
I20260805 22:44:46.822806 11299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69e3bc3b1dbe48d8bb6c2fb884b3dec8" candidate_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb"
I20260805 22:44:46.822898 11299 raft_consensus.cc:3037] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:46.823592 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.816549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:40927 (local address 127.23.197.66:33207)
0805 22:44:46.816608 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:46.816610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:46.816675 (+ 65us) server_negotiation.cc:407] Connection header received
0805 22:44:46.816973 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:46.816974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:46.816984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:46.817016 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:46.818403 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.819114 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.822289 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.822853 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.822918 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.822927 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:46.823486 (+ 559us) server_negotiation.cc:299] Negotiation successful
0805 22:44:46.823528 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260805 22:44:46.823724 11299 raft_consensus.cc:2454] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3a7ff2a92454ff98d9079d920a2a88a in term 1.
I20260805 22:44:46.823827 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.816309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:33207 (local address 127.23.197.65:40927)
0805 22:44:46.816616 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:46.816626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:46.816884 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:46.817667 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:46.817668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:46.817670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:46.817739 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.817740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.819387 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.819388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.822179 (+ 2791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:46.822184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.823324 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.823325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:46.823395 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.823398 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:44:46.823685 (+ 287us) client_negotiation.cc:240] Negotiation successful
0805 22:44:46.823740 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":142,"threads_started":1}
I20260805 22:44:46.823946 10821 leader_election.cc:302] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8161d0c78c1f44e69094ed5ff2ced1eb, b3a7ff2a92454ff98d9079d920a2a88a; no voters:
I20260805 22:44:46.824023 11441 raft_consensus.cc:2781] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:44:46.824209 11441 raft_consensus.cc:686] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [term 1 LEADER]: Becoming Leader. State: Replica: b3a7ff2a92454ff98d9079d920a2a88a, State: Running, Role: LEADER
I20260805 22:44:46.824206 11083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69e3bc3b1dbe48d8bb6c2fb884b3dec8" candidate_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd80be5ebcb4664ba45c341352faf7e" is_pre_election: true
I20260805 22:44:46.824340 11083 raft_consensus.cc:2454] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3a7ff2a92454ff98d9079d920a2a88a in term 0.
I20260805 22:44:46.824330 11441 consensus_queue.cc:237] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } }
I20260805 22:44:46.824440 11082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69e3bc3b1dbe48d8bb6c2fb884b3dec8" candidate_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd80be5ebcb4664ba45c341352faf7e"
I20260805 22:44:46.824512 11082 raft_consensus.cc:3037] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:46.825397 11082 raft_consensus.cc:2454] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3a7ff2a92454ff98d9079d920a2a88a in term 1.
I20260805 22:44:46.825667 10691 catalog_manager.cc:5495] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a reported cstate change: term changed from 0 to 1, leader changed from <none> to b3a7ff2a92454ff98d9079d920a2a88a (127.23.197.65). New cstate: current_term: 1 leader_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3a7ff2a92454ff98d9079d920a2a88a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 38969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 } health_report { overall_health: UNKNOWN } } }
W20260805 22:44:46.851359 11168 tablet.cc:2369] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:44:46.884336 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.880859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.23.197.65:38969)
0805 22:44:46.880904 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:46.880907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:46.880917 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:44:46.881012 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:46.881013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:46.881035 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:46.881125 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:44:46.881357 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.882086 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.883201 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:46.883492 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:46.883559 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:46.883766 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:44:46.883898 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:44:46.883975 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:46.884232 (+ 257us) server_negotiation.cc:299] Negotiation successful
0805 22:44:46.884254 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:44:46.888078 11299 raft_consensus.cc:1261] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 1 FOLLOWER]: Refusing update from remote peer b3a7ff2a92454ff98d9079d920a2a88a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:44:46.888437 11449 consensus_queue.cc:1035] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 46455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:44:46.890182 11082 raft_consensus.cc:1261] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [term 1 FOLLOWER]: Refusing update from remote peer b3a7ff2a92454ff98d9079d920a2a88a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:44:46.890441 11449 consensus_queue.cc:1035] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P b3a7ff2a92454ff98d9079d920a2a88a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd80be5ebcb4664ba45c341352faf7e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:44:46.894539 11460 mvcc.cc:205] Tried to move back new op lower bound from 7315332656690888704 to 7315332656432996352. Current Snapshot: MvccSnapshot[applied={T|T < 7315332656690888704}]
I20260805 22:44:46.895414 10691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:44:46.895504 10691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:44:46.896199 10691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03462265d01a4fb1978873dc83c77030: Sending DeleteTablet for 3 replicas of tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8
I20260805 22:44:46.896696 11044 tablet_service.cc:1514] Processing DeleteTablet for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:44:46 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:48082
I20260805 22:44:46.896991 11468 tablet_replica.cc:330] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: stopping tablet replica
I20260805 22:44:46.897040 10877 tablet_service.cc:1514] Processing DeleteTablet for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:44:46 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:51638
I20260805 22:44:46.897221 11468 raft_consensus.cc:2229] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:44:46.897279 11249 tablet_service.cc:1514] Processing DeleteTablet for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:44:46 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:44468
I20260805 22:44:46.897326 11468 raft_consensus.cc:2258] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:44:46.897713 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 10750
I20260805 22:44:46.909595 11468 ts_tablet_manager.cc:1905] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:44:46.911648 11468 ts_tablet_manager.cc:1918] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:44:46.911811 11468 log.cc:1196] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/69e3bc3b1dbe48d8bb6c2fb884b3dec8
I20260805 22:44:46.912182 11468 ts_tablet_manager.cc:1939] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P efd80be5ebcb4664ba45c341352faf7e: Deleting consensus metadata
I20260805 22:44:46.912704 10673 catalog_manager.cc:4841] TS efd80be5ebcb4664ba45c341352faf7e (127.23.197.66:33207): tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8 (table test-table [id=41398f624f614ca5a2c58e17f9c71f08]) successfully deleted
I20260805 22:44:46.912858 11470 tablet_replica.cc:330] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: stopping tablet replica
I20260805 22:44:46.913002 11470 raft_consensus.cc:2229] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:44:46.913113 11470 pending_rounds.cc:62] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: Trying to abort 2 pending ops.
I20260805 22:44:46.913160 11470 pending_rounds.cc:68] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7315332656690888704 op_type: WRITE_OP write_request { tablet_id: "69e3bc3b1dbe48d8bb6c2fb884b3dec8" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\212\374\316\323\006\"8\n\ntest-admin\022*\n 41398f624f614ca5a2c58e17f9c71f08\020\001\030\001 \001(\001" signature: "<redacted>""\"\013\375n\343]\276\006D&%}\355\205\031Q4\006x\377\322C\3066\313\374\023\314\32125\327\271\230\251\017\266\274r\306\022\240b\tRlT/\224U<hI\267\010\370\t\236!\376&\315}\177" signing_key_seq_num: 0 } } request_id { client_id: "4225e118e75e411d959cd4180292f434" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260805 22:44:46.913470 11470 pending_rounds.cc:68] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315332656432996352 op_type: NO_OP noop_request { }
I20260805 22:44:46.913548 11470 raft_consensus.cc:2866] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260805 22:44:46.913589 11470 raft_consensus.cc:2258] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260805 22:44:46.916558 10675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:38969 (error 108)
I20260805 22:44:46.916792 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 10983
I20260805 22:44:46.922659 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.917114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:38969 (local address 127.0.0.1:51658)
0805 22:44:46.922552 (+ 5438us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:46.922616 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:38969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5399}
W20260805 22:44:46.925369 11459 consensus_queue.cc:1159] T 69e3bc3b1dbe48d8bb6c2fb884b3dec8 P 8161d0c78c1f44e69094ed5ff2ced1eb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8161d0c78c1f44e69094ed5ff2ced1eb" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
W20260805 22:44:46.928493 10675 catalog_manager.cc:4584] TS b3a7ff2a92454ff98d9079d920a2a88a (127.23.197.65:38969): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69e3bc3b1dbe48d8bb6c2fb884b3dec8: Network error: Client connection negotiation failed: client connection to 127.23.197.65:38969: connect: Connection refused (error 111)
I20260805 22:44:46.929373 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 11171
I20260805 22:44:46.939361 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 10650
I20260805 22:44:46.939437 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:46.939314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:46455 (local address 127.0.0.1:44482)
0805 22:44:46.939369 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:46.939395 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.67:46455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260805 22:44:46.950929 24341 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260805 22:44:47.295920 24341 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260805 22:44:47.296017 24341 mini_ranger.cc:66] Stopping Ranger...
I20260805 22:44:47.650883 24341 mini_ranger.cc:68] Stopped Ranger
2026-08-05 22:44:47.651 UTC [9244] LOG: received smart shutdown request
2026-08-05 22:44:47.655 UTC [9244] LOG: background worker "logical replication launcher" (PID 9257) exited with exit code 1
2026-08-05 22:44:47.656 UTC [9251] LOG: shutting down
2026-08-05 22:44:47.845 UTC [9244] LOG: database system is shut down
2026-08-05T22:44:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52048 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:44:48 dist-test-slave-g170 krb5kdc[11485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:44:48 dist-test-slave-g170 krb5kdc[11485](info): set up 2 sockets
Aug 05 22:44:48 dist-test-slave-g170 krb5kdc[11485](info): commencing operation
krb5kdc: starting...
W20260805 22:44:49.255722 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:44:49.391325 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40409
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:40409,127.23.197.125:40219,127.23.197.124:38879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:49.475479 11505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:49.475728 11505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:49.475764 11505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:49.476892 11505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:44:49.476959 11505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:49.476984 11505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:49.477005 11505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:44:49.477036 11505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:44:49.478531 11505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:40409,127.23.197.125:40219,127.23.197.124:38879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:40409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:49.478822 11505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:49.479051 11505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:49.484038 11511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:49.484035 11512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:49.484117 11505 server_base.cc:1034] running on GCE node
W20260805 22:44:49.484035 11515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:49.484494 11505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:49.484835 11505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:49.486016 11505 hybrid_clock.cc:648] HybridClock initialized: now 1785969889485995 us; error 41 us; skew 500 ppm
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:49.488718 11505 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:44:49.489250 11505 webserver.cc:460] Webserver started at http://127.23.197.126:36261/ using document root <none> and password file <none>
I20260805 22:44:49.489501 11505 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:49.489573 11505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:49.489729 11505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:44:49.490828 11505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "a76981d6cf064ea6bcd0465058c8b932"
format_stamp: "Formatted at 2026-08-05 22:44:49 on dist-test-slave-g170"
server_key: "6f9d6c411580b2e23a49d55ed43184a7"
server_key_iv: "8e21e63ad3d74501fae8856b690fdd30"
server_key_version: "encryptionkey@0"
I20260805 22:44:49.491207 11505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "a76981d6cf064ea6bcd0465058c8b932"
format_stamp: "Formatted at 2026-08-05 22:44:49 on dist-test-slave-g170"
server_key: "6f9d6c411580b2e23a49d55ed43184a7"
server_key_iv: "8e21e63ad3d74501fae8856b690fdd30"
server_key_version: "encryptionkey@0"
I20260805 22:44:49.492652 11505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260805 22:44:49.493583 11525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:49.493907 11505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:44:49.494050 11505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "a76981d6cf064ea6bcd0465058c8b932"
format_stamp: "Formatted at 2026-08-05 22:44:49 on dist-test-slave-g170"
server_key: "6f9d6c411580b2e23a49d55ed43184a7"
server_key_iv: "8e21e63ad3d74501fae8856b690fdd30"
server_key_version: "encryptionkey@0"
I20260805 22:44:49.494135 11505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:49.504328 11505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:49.506333 11505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:49.506505 11505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:49.510890 11505 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:40409
I20260805 22:44:49.510927 11592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:40409 every 8 connection(s)
I20260805 22:44:49.511301 11505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:44:49.512234 11593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:49.513929 11593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:44:49.517156 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 11505
I20260805 22:44:49.517275 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:44:49.517448 24341 external_mini_cluster.cc:1428] Setting key 45b7466b3faa98c81063ff74fe1bae8d
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:49.525934 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.514603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40409 (local address 127.0.0.1:44562)
0805 22:44:49.514859 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.514905 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:49.515447 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:49.515749 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:49.515753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:49.516038 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:49.516237 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.516243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.517745 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.517746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.518612 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.518615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.519357 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.519359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.519455 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.520077 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:49.520090 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:49.521923 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:49.524254 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.524261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.524264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.524487 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.524825 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.524826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.524827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.524931 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.525393 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:49.525399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:49.525460 (+ 61us) client_negotiation.cc:769] Sending connection context
0805 22:44:49.525539 (+ 79us) client_negotiation.cc:240] Negotiation successful
0805 22:44:49.525709 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":101,"threads_started":1}
I20260805 22:44:49.525941 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.514675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.23.197.126:40409)
0805 22:44:49.514872 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.514882 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.515018 (+ 136us) server_negotiation.cc:407] Connection header received
0805 22:44:49.515617 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.515619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.515636 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.515742 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.516346 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.517594 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.518720 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.519241 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.519366 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.522026 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.522038 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.522039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.522047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.524158 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.524585 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.524588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.524589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.524707 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.525004 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.525009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.525009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.525316 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.525407 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.525614 (+ 207us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.525797 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.23.197.125@KRBTEST.COM; defaulting to no policy
I20260805 22:44:49.527262 11593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:44:49.527668 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.527531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:40219 (local address 127.0.0.1:38274)
0805 22:44:49.527586 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.527633 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260805 22:44:49.527894 11533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
I20260805 22:44:49.528368 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.528273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:40219 (local address 127.0.0.1:38286)
0805 22:44:49.528308 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.528340 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260805 22:44:49.528514 11593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.125:40219: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab.
Entry for principal kudu/127.23.197.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab.
I20260805 22:44:49.547887 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.125:40219
--webserver_interface=127.23.197.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab
--principal=kudu/127.23.197.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:40409,127.23.197.125:40219,127.23.197.124:38879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:44:49.577767 11593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } attempt: 1
I20260805 22:44:49.578351 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.578184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:40219 (local address 127.0.0.1:38294)
0805 22:44:49.578258 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.578304 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260805 22:44:49.578749 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.578658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:40219 (local address 127.0.0.1:38298)
0805 22:44:49.578705 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.578731 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260805 22:44:49.578869 11593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.125:40219: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
W20260805 22:44:49.632117 11602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:49.632314 11602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:49.632337 11602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:49.633507 11602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:44:49.633548 11602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:49.633562 11602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:49.633575 11602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:44:49.633587 11602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:44:49.634976 11602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:40409,127.23.197.125:40219,127.23.197.124:38879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab
--principal=kudu/127.23.197.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.125:40219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.197.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:49.635159 11602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:49.635324 11602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:49.640223 11615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:49.640223 11613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:49.640275 11602 server_base.cc:1034] running on GCE node
W20260805 22:44:49.640362 11610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:49.640638 11602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:49.640959 11602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:49.642125 11602 hybrid_clock.cc:648] HybridClock initialized: now 1785969889642116 us; error 31 us; skew 500 ppm
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:49.644707 11602 init.cc:377] Logged in from keytab as kudu/127.23.197.125@KRBTEST.COM (short username kudu)
I20260805 22:44:49.645200 11602 webserver.cc:460] Webserver started at http://127.23.197.125:44303/ using document root <none> and password file <none>
I20260805 22:44:49.645452 11602 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:49.645529 11602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:49.645690 11602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:44:49.646739 11602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/data/instance:
uuid: "0616aa97417e401987867164b97cc67b"
format_stamp: "Formatted at 2026-08-05 22:44:49 on dist-test-slave-g170"
server_key: "909c9994e0eacbc62297727abc5f92d8"
server_key_iv: "4455a00cea801e3dacad91c50f72dfee"
server_key_version: "encryptionkey@0"
I20260805 22:44:49.647125 11602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/wal/instance:
uuid: "0616aa97417e401987867164b97cc67b"
format_stamp: "Formatted at 2026-08-05 22:44:49 on dist-test-slave-g170"
server_key: "909c9994e0eacbc62297727abc5f92d8"
server_key_iv: "4455a00cea801e3dacad91c50f72dfee"
server_key_version: "encryptionkey@0"
I20260805 22:44:49.647078 11593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } attempt: 2
I20260805 22:44:49.647504 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.647377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:40219 (local address 127.0.0.1:38308)
0805 22:44:49.647444 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.647477 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260805 22:44:49.647893 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.647799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:40219 (local address 127.0.0.1:38312)
0805 22:44:49.647844 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.647875 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260805 22:44:49.647990 11593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.125:40219: Network error: Client connection negotiation failed: client connection to 127.23.197.125:40219: connect: Connection refused (error 111)
I20260805 22:44:49.648692 11602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:44:49.649488 11625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:49.649667 11602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:49.649765 11602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/wal
uuid: "0616aa97417e401987867164b97cc67b"
format_stamp: "Formatted at 2026-08-05 22:44:49 on dist-test-slave-g170"
server_key: "909c9994e0eacbc62297727abc5f92d8"
server_key_iv: "4455a00cea801e3dacad91c50f72dfee"
server_key_version: "encryptionkey@0"
I20260805 22:44:49.649844 11602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:49.662115 11602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:49.664399 11602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:49.664572 11602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:49.669179 11602 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.125:40219
I20260805 22:44:49.669234 11713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.125:40219 every 8 connection(s)
I20260805 22:44:49.669588 11602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
I20260805 22:44:49.670452 11714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:49.672039 11714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:44:49.673382 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 11602
I20260805 22:44:49.673498 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-1/wal/instance
I20260805 22:44:49.673754 24341 external_mini_cluster.cc:1428] Setting key bab6b3becac0e1ec08bd58509675b8f2
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:49.682228 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.672797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44566 (local address 127.23.197.126:40409)
0805 22:44:49.672867 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.672869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.673055 (+ 186us) server_negotiation.cc:407] Connection header received
0805 22:44:49.673758 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.673759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.673773 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.673843 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.674696 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.675180 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.676369 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.676587 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.676861 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.679119 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.679121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.679122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.679130 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.680751 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.681233 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.681235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.681235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.681302 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.681560 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.681561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.681561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.681684 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.681758 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.682037 (+ 279us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.682126 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260805 22:44:49.682439 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.672688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40409 (local address 127.0.0.1:44566)
0805 22:44:49.672923 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.672971 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:49.673624 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:49.673923 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:49.673928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:49.674312 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:49.674595 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.674600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.675273 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.675274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.676265 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.676268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.676868 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.676870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.677021 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.677601 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:49.677611 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:49.679010 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:49.680901 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.680909 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.680912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.681151 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.681373 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.681374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.681374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.681488 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.681762 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:49.681770 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:49.681906 (+ 136us) client_negotiation.cc:769] Sending connection context
0805 22:44:49.682015 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 22:44:49.682191 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.23.197.124@KRBTEST.COM; defaulting to no policy
I20260805 22:44:49.683002 11714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.124@KRBTEST.COM" created.
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:44:49.690955 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.683306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:40219 (local address 127.0.0.1:38322)
0805 22:44:49.683361 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.683378 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:49.683411 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:49.683731 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:49.683732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:49.683866 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:49.683910 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.683911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.684834 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.684834 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.685375 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.685377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.685863 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.685864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.685930 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.686365 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:49.686370 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:49.687638 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:49.689933 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.689934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.689934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.690098 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.690335 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.690335 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.690336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.690391 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.690731 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:49.690732 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:49.690769 (+ 37us) client_negotiation.cc:769] Sending connection context
0805 22:44:49.690817 (+ 48us) client_negotiation.cc:240] Negotiation successful
0805 22:44:49.690888 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260805 22:44:49.691221 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.683390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.23.197.125:40219)
0805 22:44:49.683564 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.683568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.683605 (+ 37us) server_negotiation.cc:407] Connection header received
0805 22:44:49.683627 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.683629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.683641 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.683715 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.683994 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.684699 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.685465 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.685787 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.685867 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.687736 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.687745 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.687745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.687754 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.689852 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.690185 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.690188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.690189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.690263 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.690458 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.690459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.690459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.690654 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.690745 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.690858 (+ 113us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.690970 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260805 22:44:49.691901 11714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:44:49.692256 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.692142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59258)
0805 22:44:49.692192 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.692238 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260805 22:44:49.692391 11632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
I20260805 22:44:49.692879 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.692747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59266)
0805 22:44:49.692827 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.692853 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260805 22:44:49.693024 11714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38879: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab.
Entry for principal kudu/127.23.197.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab.
I20260805 22:44:49.704589 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.124:38879
--webserver_interface=127.23.197.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab
--principal=kudu/127.23.197.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:40409,127.23.197.125:40219,127.23.197.124:38879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:44:49.739130 11593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } attempt: 3
I20260805 22:44:49.742259 11714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } attempt: 1
I20260805 22:44:49.742704 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.742572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59278)
0805 22:44:49.742630 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.742669 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260805 22:44:49.743127 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.743026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59284)
0805 22:44:49.743071 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.743097 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260805 22:44:49.743259 11714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38879: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:44:49.747072 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.739618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.23.197.125:40219)
0805 22:44:49.739715 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.739717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.739728 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:44:49.739758 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.739759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.739770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.739843 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.740195 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.740733 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.741509 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.741734 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.741816 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.744350 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.744352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.744352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.744358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.745906 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.746313 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.746314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.746314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.746373 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.746577 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.746577 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.746578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.746685 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.746746 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.746947 (+ 201us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.747017 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 22:44:49.747076 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.739541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:40219 (local address 127.0.0.1:38326)
0805 22:44:49.739604 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.739622 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:49.739694 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:49.739851 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:49.739852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:49.740028 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:49.740097 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.740098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.740870 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.740871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.741411 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.741412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.741812 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.741813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.741896 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.742416 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:49.742423 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:49.744246 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:49.745991 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.745992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.745993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.746213 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.746445 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.746446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.746447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.746515 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.746754 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:49.746755 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:49.746810 (+ 55us) client_negotiation.cc:769] Sending connection context
0805 22:44:49.746885 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 22:44:49.746982 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260805 22:44:49.747677 11593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:44:49.748030 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.747918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59290)
0805 22:44:49.747970 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.748005 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260805 22:44:49.748415 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.748324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59300)
0805 22:44:49.748370 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.748398 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260805 22:44:49.748514 11593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38879: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
I20260805 22:44:49.779673 11593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } attempt: 1
I20260805 22:44:49.780212 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.780057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59316)
0805 22:44:49.780129 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.780176 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260805 22:44:49.780643 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.780541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59324)
0805 22:44:49.780591 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.780620 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260805 22:44:49.780778 11593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38879: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
W20260805 22:44:49.795593 11721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:49.795894 11721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:49.795931 11721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:49.798115 11721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:44:49.798192 11721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:49.798216 11721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:49.798236 11721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:44:49.798254 11721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:44:49.800361 11721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:40409,127.23.197.125:40219,127.23.197.124:38879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab
--principal=kudu/127.23.197.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.124:38879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.197.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:49.800678 11721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:49.800992 11721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:49.806124 11731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:49.806260 11730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:49.806151 11734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:49.806275 11721 server_base.cc:1034] running on GCE node
I20260805 22:44:49.806896 11721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:49.807246 11721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:49.808481 11721 hybrid_clock.cc:648] HybridClock initialized: now 1785969889808483 us; error 48 us; skew 500 ppm
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:49.811115 11721 init.cc:377] Logged in from keytab as kudu/127.23.197.124@KRBTEST.COM (short username kudu)
I20260805 22:44:49.811427 11714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } attempt: 2
I20260805 22:44:49.811673 11721 webserver.cc:460] Webserver started at http://127.23.197.124:44513/ using document root <none> and password file <none>
I20260805 22:44:49.811889 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.811737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59334)
0805 22:44:49.811832 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.811859 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260805 22:44:49.811915 11721 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:49.811987 11721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:49.812147 11721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:44:49.812290 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.812173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59348)
0805 22:44:49.812232 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.812266 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260805 22:44:49.812413 11714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38879: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38879: connect: Connection refused (error 111)
I20260805 22:44:49.813447 11721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/data/instance:
uuid: "ca6ec773f9294550b4f03f706544c7d1"
format_stamp: "Formatted at 2026-08-05 22:44:49 on dist-test-slave-g170"
server_key: "ae6de6d0b85cb2162195ff436f239dcd"
server_key_iv: "5d3aaf5fd46b5772d7a6d32c787c40e1"
server_key_version: "encryptionkey@0"
I20260805 22:44:49.813766 11721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/wal/instance:
uuid: "ca6ec773f9294550b4f03f706544c7d1"
format_stamp: "Formatted at 2026-08-05 22:44:49 on dist-test-slave-g170"
server_key: "ae6de6d0b85cb2162195ff436f239dcd"
server_key_iv: "5d3aaf5fd46b5772d7a6d32c787c40e1"
server_key_version: "encryptionkey@0"
I20260805 22:44:49.815083 11721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:44:49.815857 11744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:49.816139 11721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:49.816237 11721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/wal
uuid: "ca6ec773f9294550b4f03f706544c7d1"
format_stamp: "Formatted at 2026-08-05 22:44:49 on dist-test-slave-g170"
server_key: "ae6de6d0b85cb2162195ff436f239dcd"
server_key_iv: "5d3aaf5fd46b5772d7a6d32c787c40e1"
server_key_version: "encryptionkey@0"
I20260805 22:44:49.816293 11721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:49.842022 11721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:49.844434 11721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:49.844563 11721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:49.848842 11721 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.124:38879
I20260805 22:44:49.848912 11825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.124:38879 every 8 connection(s)
I20260805 22:44:49.849274 11721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
I20260805 22:44:49.850349 11826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:49.850500 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 11721
I20260805 22:44:49.850615 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785969725087118-24341-0/minicluster-data/master-2/wal/instance
I20260805 22:44:49.850863 24341 external_mini_cluster.cc:1428] Setting key 8447ccfa9276983c0bbfd5694509b7e7
I20260805 22:44:49.853391 11826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:44:49.855921 11593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } attempt: 2
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
I20260805 22:44:49.863641 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.851432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59362 (local address 127.23.197.124:38879)
0805 22:44:49.851596 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.851604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.851631 (+ 27us) server_negotiation.cc:407] Connection header received
0805 22:44:49.852139 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.852145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.852219 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.852380 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.853383 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.854472 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.855527 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.856145 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.856262 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.859717 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.859725 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.859726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.859733 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.861788 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.862184 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.862187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.862188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.862277 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.862516 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.862518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.862518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.862870 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.863083 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.863237 (+ 154us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.863388 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260805 22:44:49.863736 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.854161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.23.197.126:40409)
0805 22:44:49.854237 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.854239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.854455 (+ 216us) server_negotiation.cc:407] Connection header received
0805 22:44:49.854514 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.854515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.854527 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.854580 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.855181 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.855709 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.856883 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.857145 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.857201 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.860075 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.860077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.860077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.860085 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.861704 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.862533 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.862534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.862535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.862596 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.862905 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.862906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.862906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.863116 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.863191 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.863503 (+ 312us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.863608 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 22:44:49.864081 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.854127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:40409 (local address 127.0.0.1:44578)
0805 22:44:49.854331 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.854375 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:49.854451 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:49.854619 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:49.854622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:49.855008 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:49.855098 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.855101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.855809 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.855810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.856725 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.856728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.857236 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.857237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.857316 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.857886 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:49.857897 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:49.859921 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:49.861796 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.861803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.861805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.862088 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.862751 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.862752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.862752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.862804 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.863255 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:49.863262 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:49.863325 (+ 63us) client_negotiation.cc:769] Sending connection context
0805 22:44:49.863848 (+ 523us) client_negotiation.cc:240] Negotiation successful
0805 22:44:49.863964 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":148,"threads_started":1}
I20260805 22:44:49.864822 11826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:44:49.865536 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.856178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59370)
0805 22:44:49.856224 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.856235 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:49.856267 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:49.856535 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:49.856537 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:49.856713 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:49.856755 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.856757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.857650 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.857651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.858527 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.858531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.859027 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.859028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.859073 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.859524 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:49.859530 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:49.861214 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:49.864441 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.864442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.864443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.864658 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.864868 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.864869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.864869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.864923 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.865233 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:49.865234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:49.865288 (+ 54us) client_negotiation.cc:769] Sending connection context
0805 22:44:49.865352 (+ 64us) client_negotiation.cc:240] Negotiation successful
0805 22:44:49.865451 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260805 22:44:49.866046 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.856245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.23.197.124:38879)
0805 22:44:49.856400 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.856401 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.856408 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:44:49.856424 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.856424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.856434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.856507 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.856872 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.857530 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.858629 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.858941 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.859012 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.861314 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.861315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.861316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.861322 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.864337 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.864721 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.864722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.864723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.864808 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.864997 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.864998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.864998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.865134 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.865855 (+ 721us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.865895 (+ 40us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.865965 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260805 22:44:49.868554 11593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932: Bootstrap starting.
I20260805 22:44:49.869514 11593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932: Neither blocks nor log segments found. Creating new log.
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:44:49.869874 11593 log.cc:824] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932: Log is configured to *not* fsync() on all Append() calls
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:49.871287 11593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932: No bootstrap required, opened a new log
I20260805 22:44:49.873315 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.865133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:40219 (local address 127.0.0.1:38336)
0805 22:44:49.865259 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.865272 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:49.865331 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:49.865468 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:49.865468 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:49.865676 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:49.865734 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.865736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.866736 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.866737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.867302 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.867303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.867917 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.867918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.868016 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.868510 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:49.868516 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:49.870147 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:49.872146 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.872147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.872148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.872333 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.872560 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.872561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.872561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.872630 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.872971 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:49.872972 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:49.873019 (+ 47us) client_negotiation.cc:769] Sending connection context
0805 22:44:49.873108 (+ 89us) client_negotiation.cc:240] Negotiation successful
0805 22:44:49.873221 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260805 22:44:49.873315 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.865212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38336 (local address 127.23.197.125:40219)
0805 22:44:49.865266 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.865269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.865313 (+ 44us) server_negotiation.cc:407] Connection header received
0805 22:44:49.865388 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.865389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.865400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.865463 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.865844 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.866622 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.867438 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.867797 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.867839 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.870257 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.870259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.870259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.870268 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.872041 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.872428 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.872430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.872430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.872496 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.872713 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.872714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.872714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.872892 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.872966 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.873137 (+ 171us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.873213 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:44:49.873708 11593 raft_consensus.cc:348] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } }
I20260805 22:44:49.873925 11593 raft_consensus.cc:374] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:49.873912 11826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:44:49.874014 11593 raft_consensus.cc:729] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a76981d6cf064ea6bcd0465058c8b932, State: Initialized, Role: FOLLOWER
I20260805 22:44:49.874136 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.864621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.23.197.126:40409)
0805 22:44:49.864845 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.864848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.864857 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:44:49.864878 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.864879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.864889 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.864954 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.865625 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.866439 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.867372 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.867680 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.867754 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.870804 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.870806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.870806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.870814 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.872424 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.872832 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.872833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.872834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.872892 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.873119 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.873119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.873120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.873258 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.873876 (+ 618us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.873922 (+ 46us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.874010 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199}
I20260805 22:44:49.874305 11593 consensus_queue.cc:260] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } }
I20260805 22:44:49.874763 11593 sys_catalog.cc:564] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:44:49.874724 11835 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:49.874861 11835 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [sys.catalog]: This master's current role is: FOLLOWER
W20260805 22:44:49.876789 11846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:44:49.876910 11846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
I20260805 22:44:49.877751 11505 master_runner.cc:186] Error getting master registration for 127.23.197.125:40219: OK, instance_id { permanent_uuid: "0616aa97417e401987867164b97cc67b" instance_seqno: 1785969889667942 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 22:44:49.881356 11505 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 22:44:49.884375 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.874211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59382)
0805 22:44:49.874322 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.874331 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:49.874366 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:49.876315 (+ 1949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:49.876316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:49.876473 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:49.876511 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.876513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.877426 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.877427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.878248 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.878251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.878768 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.878769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.878816 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.879281 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:49.879286 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:49.881285 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:49.883202 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.883203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.883204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.883426 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.883652 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.883653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.883654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.883746 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.884086 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:49.884087 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:49.884126 (+ 39us) client_negotiation.cc:769] Sending connection context
0805 22:44:49.884173 (+ 47us) client_negotiation.cc:240] Negotiation successful
0805 22:44:49.884277 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260805 22:44:49.884410 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.874701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.23.197.125:40219)
0805 22:44:49.874847 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.874849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.874858 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:44:49.874877 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.874878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.874888 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.874935 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.875459 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.876212 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.877267 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.877539 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.877640 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.881370 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.881372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.881372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.881381 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.883195 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.883577 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.883578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.883579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.883639 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.883859 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.883860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.883861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.883980 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.884195 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.884247 (+ 52us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.884321 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260805 22:44:49.884961 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.874292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.23.197.124:38879)
0805 22:44:49.876178 (+ 1886us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.876180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.876189 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:44:49.876211 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.876211 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.876224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.876576 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.876598 (+ 22us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.877345 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.878366 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.878675 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.878973 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.881379 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.881380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.881380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.881387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.883088 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.883513 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.883514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.883515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.883581 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.883796 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.883797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.883798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.884001 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.884727 (+ 726us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.884827 (+ 100us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.884907 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1864}
I20260805 22:44:49.886509 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:44:49.886740 11826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1: Bootstrap starting.
I20260805 22:44:49.887409 11826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:49.887796 11826 log.cc:824] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:49.889503 11826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1: No bootstrap required, opened a new log
I20260805 22:44:49.891136 11826 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } }
I20260805 22:44:49.891288 11826 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:49.891330 11826 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6ec773f9294550b4f03f706544c7d1, State: Initialized, Role: FOLLOWER
I20260805 22:44:49.891471 11826 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } }
I20260805 22:44:49.891815 11826 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:44:49.892056 11860 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:49.892144 11860 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [sys.catalog]: This master's current role is: FOLLOWER
W20260805 22:44:49.894207 11875 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:44:49.894348 11875 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:44:49.896158 11721 master_runner.cc:186] Error getting master registration for 127.23.197.125:40219: OK, instance_id { permanent_uuid: "0616aa97417e401987867164b97cc67b" instance_seqno: 1785969889667942 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 22:44:49.898648 11721 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 22:44:49.899322 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.888246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44600 (local address 127.23.197.126:40409)
0805 22:44:49.888287 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.888289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.888297 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:44:49.888327 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.888328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.888338 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.888404 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.889166 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.889891 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.891026 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.891293 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.891377 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.893099 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.893101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.893101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.893111 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.895056 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.895454 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.895455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.895456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.895515 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.895715 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.895716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.895716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.895847 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.895938 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.899128 (+ 3190us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.899210 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:44:49.901885 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.888085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.23.197.125:40219)
0805 22:44:49.888130 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.888133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.888411 (+ 278us) server_negotiation.cc:407] Connection header received
0805 22:44:49.888508 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.888509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.888519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.888548 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.893162 (+ 4614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.894782 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.896716 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.897034 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.897083 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.898804 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.898806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.898806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.898814 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.900557 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.901013 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.901014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.901015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.901074 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.901284 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.901285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.901286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.901413 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.901485 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.901695 (+ 210us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.901789 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260805 22:44:49.903568 11714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } attempt: 3
I20260805 22:44:49.904464 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.888573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59388 (local address 127.23.197.124:38879)
0805 22:44:49.888618 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.888619 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.893539 (+ 4920us) server_negotiation.cc:407] Connection header received
0805 22:44:49.893605 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.893606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.893617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.893665 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.897597 (+ 3932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.898346 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.899354 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.899670 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.899722 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.901361 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.901363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.901363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.901371 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.903078 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.903582 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.903583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.903583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.903640 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.903866 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.903867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.903867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.903992 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.904085 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.904289 (+ 204us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.904370 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 05 22:44:49 dist-test-slave-g170 krb5kdc[11485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
I20260805 22:44:49.912148 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.903861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38879 (local address 127.0.0.1:59392)
0805 22:44:49.903912 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:49.903922 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:49.903963 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:49.904117 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:49.904118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:49.904279 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:49.904330 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.904331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.905408 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.905409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.906226 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:49.906228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.906811 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.906812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:49.906879 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.907344 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:49.907349 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:49.908859 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:49.911016 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.911017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.911017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.911246 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.911469 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:49.911469 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:49.911470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:49.911536 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:49.911827 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:49.911828 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:49.911874 (+ 46us) client_negotiation.cc:769] Sending connection context
0805 22:44:49.911948 (+ 74us) client_negotiation.cc:240] Negotiation successful
0805 22:44:49.912045 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260805 22:44:49.912160 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:49.903905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.23.197.124:38879)
0805 22:44:49.903944 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:49.903946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:49.903954 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:44:49.904022 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:49.904022 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:49.904032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:49.904074 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:49.904422 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.905315 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.906322 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:49.906659 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:49.906743 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:49.908960 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:49.908961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:49.908961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:49.908968 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:49.910626 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.911327 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.911328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.911329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.911392 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:49.911606 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:49.911606 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:49.911607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:49.911750 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:49.911819 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:49.911986 (+ 167us) server_negotiation.cc:299] Negotiation successful
0805 22:44:49.912070 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260805 22:44:49.914341 11714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b: Bootstrap starting.
I20260805 22:44:49.915107 11714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:49.915444 11714 log.cc:824] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:49.916675 11714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b: No bootstrap required, opened a new log
I20260805 22:44:49.918888 11714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } }
I20260805 22:44:49.919037 11714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:49.919073 11714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0616aa97417e401987867164b97cc67b, State: Initialized, Role: FOLLOWER
I20260805 22:44:49.919193 11714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } }
I20260805 22:44:49.919549 11714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [sys.catalog]: configured and running, proceeding with master startup.
W20260805 22:44:49.921455 11894 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:44:49.921564 11894 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:44:49.923312 11602 master_runner.cc:263] No leader master found from master 0616aa97417e401987867164b97cc67b
I20260805 22:44:49.923359 11602 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 22:44:49.924911 11880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:49.925102 11880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:44:49.993675 11835 raft_consensus.cc:482] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:44:49.993821 11835 raft_consensus.cc:504] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } }
I20260805 22:44:49.994081 11835 leader_election.cc:288] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0616aa97417e401987867164b97cc67b (127.23.197.125:40219), ca6ec773f9294550b4f03f706544c7d1 (127.23.197.124:38879)
I20260805 22:44:49.994482 11672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a76981d6cf064ea6bcd0465058c8b932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0616aa97417e401987867164b97cc67b" is_pre_election: true
I20260805 22:44:49.994647 11672 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a76981d6cf064ea6bcd0465058c8b932 in term 0.
I20260805 22:44:49.995016 11533 leader_election.cc:302] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0616aa97417e401987867164b97cc67b, a76981d6cf064ea6bcd0465058c8b932; no voters:
I20260805 22:44:49.995244 11835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:44:49.995311 11835 raft_consensus.cc:482] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:44:49.995337 11835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:49.996302 11835 raft_consensus.cc:504] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } }
I20260805 22:44:49.996431 11835 leader_election.cc:288] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [CANDIDATE]: Term 1 election: Requested vote from peers 0616aa97417e401987867164b97cc67b (127.23.197.125:40219), ca6ec773f9294550b4f03f706544c7d1 (127.23.197.124:38879)
I20260805 22:44:49.996757 11672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a76981d6cf064ea6bcd0465058c8b932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0616aa97417e401987867164b97cc67b"
I20260805 22:44:49.996922 11672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:49.997882 11672 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a76981d6cf064ea6bcd0465058c8b932 in term 1.
I20260805 22:44:49.998152 11533 leader_election.cc:302] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0616aa97417e401987867164b97cc67b, a76981d6cf064ea6bcd0465058c8b932; no voters:
I20260805 22:44:49.998553 11787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a76981d6cf064ea6bcd0465058c8b932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6ec773f9294550b4f03f706544c7d1" is_pre_election: true
I20260805 22:44:49.998710 11787 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a76981d6cf064ea6bcd0465058c8b932 in term 0.
I20260805 22:44:49.998826 11785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a76981d6cf064ea6bcd0465058c8b932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6ec773f9294550b4f03f706544c7d1"
I20260805 22:44:49.998884 11785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:49.999743 11785 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a76981d6cf064ea6bcd0465058c8b932 in term 1.
I20260805 22:44:50.001127 11835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:44:50.001258 11835 raft_consensus.cc:686] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [term 1 LEADER]: Becoming Leader. State: Replica: a76981d6cf064ea6bcd0465058c8b932, State: Running, Role: LEADER
I20260805 22:44:50.001345 11835 consensus_queue.cc:237] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } }
I20260805 22:44:50.001777 11835 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a76981d6cf064ea6bcd0465058c8b932. Latest consensus state: current_term: 1 leader_uuid: "a76981d6cf064ea6bcd0465058c8b932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:50.001866 11835 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:50.002354 11898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:44:50.002607 11898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:44:50.005065 11785 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [term 1 FOLLOWER]: Refusing update from remote peer a76981d6cf064ea6bcd0465058c8b932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:44:50.005499 11835 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:44:50.006932 11860 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a76981d6cf064ea6bcd0465058c8b932. Latest consensus state: current_term: 1 leader_uuid: "a76981d6cf064ea6bcd0465058c8b932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:50.007022 11860 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [sys.catalog]: This master's current role is: FOLLOWER
W20260805 22:44:50.007367 11535 tablet.cc:2369] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:44:50.007781 11672 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [term 1 FOLLOWER]: Refusing update from remote peer a76981d6cf064ea6bcd0465058c8b932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:44:50.008177 11835 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:44:50.009522 11880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a76981d6cf064ea6bcd0465058c8b932. Latest consensus state: current_term: 1 leader_uuid: "a76981d6cf064ea6bcd0465058c8b932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:50.009622 11880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:44:50.011335 11835 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a76981d6cf064ea6bcd0465058c8b932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:50.011426 11835 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:50.012019 11880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a76981d6cf064ea6bcd0465058c8b932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:50.012109 11880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:44:50.012421 11835 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a76981d6cf064ea6bcd0465058c8b932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:50.012513 11835 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932 [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:50.013010 11860 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a76981d6cf064ea6bcd0465058c8b932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a76981d6cf064ea6bcd0465058c8b932" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 40409 } } peers { permanent_uuid: "0616aa97417e401987867164b97cc67b" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 40219 } } peers { permanent_uuid: "ca6ec773f9294550b4f03f706544c7d1" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38879 } } }
I20260805 22:44:50.013095 11860 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:44:50.013447 11903 mvcc.cc:205] Tried to move back new op lower bound from 7315332669458104320 to 7315332669445947392. Current Snapshot: MvccSnapshot[applied={T|T < 7315332669458104320}]
I20260805 22:44:50.014032 11900 mvcc.cc:205] Tried to move back new op lower bound from 7315332669458104320 to 7315332669445947392. Current Snapshot: MvccSnapshot[applied={T|T < 7315332669458104320}]
I20260805 22:44:50.018898 11898 catalog_manager.cc:1347] Generated new cluster ID: 61ab33b4b2e34837ad57537f88a1ee41
I20260805 22:44:50.018965 11898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:44:50.036369 11898 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:44:50.037226 11898 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:44:50.058701 11898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a76981d6cf064ea6bcd0465058c8b932: Generated new TSK 0
I20260805 22:44:50.058976 11898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:44:50.894917 11875 catalog_manager.cc:1259] Loaded cluster ID: 61ab33b4b2e34837ad57537f88a1ee41
I20260805 22:44:50.895005 11875 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1: loading cluster ID for follower catalog manager: success
I20260805 22:44:50.895923 11875 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1: acquiring CA information for follower catalog manager: success
I20260805 22:44:50.896275 11875 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ca6ec773f9294550b4f03f706544c7d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 22:44:50.922104 11894 catalog_manager.cc:1259] Loaded cluster ID: 61ab33b4b2e34837ad57537f88a1ee41
I20260805 22:44:50.922189 11894 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b: loading cluster ID for follower catalog manager: success
I20260805 22:44:50.923097 11894 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b: acquiring CA information for follower catalog manager: success
I20260805 22:44:50.923457 11894 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0616aa97417e401987867164b97cc67b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 22:44:51.113508 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 11505
I20260805 22:44:51.126948 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 11602
I20260805 22:44:51.138302 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 11721
2026-08-05T22:44:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3161 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:44:51 dist-test-slave-g170 krb5kdc[11920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:44:51 dist-test-slave-g170 krb5kdc[11920](info): set up 2 sockets
Aug 05 22:44:51 dist-test-slave-g170 krb5kdc[11920](info): commencing operation
krb5kdc: starting...
W20260805 22:44:52.375236 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:44:52 dist-test-slave-g170 krb5kdc[11920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:44:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:44:52.505939 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:34445
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:34445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:52.588573 11935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:52.588871 11935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:52.588912 11935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:52.590090 11935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:44:52.590198 11935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:52.590224 11935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:52.590246 11935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:44:52.590276 11935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:44:52.591856 11935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:34445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:34445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:52.592155 11935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:52.592398 11935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:52.597452 11945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:52.597568 11935 server_base.cc:1034] running on GCE node
W20260805 22:44:52.597431 11947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:52.597474 11943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:52.598042 11935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:52.598358 11935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:52.599539 11935 hybrid_clock.cc:648] HybridClock initialized: now 1785969892599542 us; error 45 us; skew 500 ppm
Aug 05 22:44:52 dist-test-slave-g170 krb5kdc[11920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969892, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:52.602330 11935 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:44:52.602799 11935 webserver.cc:460] Webserver started at http://127.23.197.126:33463/ using document root <none> and password file <none>
I20260805 22:44:52.603048 11935 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:52.603118 11935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:52.603279 11935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:44:52.604418 11935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "06be464580844af6b5550f7b196a956d"
format_stamp: "Formatted at 2026-08-05 22:44:52 on dist-test-slave-g170"
server_key: "e5ab6d2fee25e7c9e419cffb54d27a99"
server_key_iv: "88e543fd9b733bb82d79ee9257014cad"
server_key_version: "encryptionkey@0"
I20260805 22:44:52.604856 11935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "06be464580844af6b5550f7b196a956d"
format_stamp: "Formatted at 2026-08-05 22:44:52 on dist-test-slave-g170"
server_key: "e5ab6d2fee25e7c9e419cffb54d27a99"
server_key_iv: "88e543fd9b733bb82d79ee9257014cad"
server_key_version: "encryptionkey@0"
I20260805 22:44:52.606247 11935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:44:52.607102 11956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:52.607316 11935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:52.607429 11935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "06be464580844af6b5550f7b196a956d"
format_stamp: "Formatted at 2026-08-05 22:44:52 on dist-test-slave-g170"
server_key: "e5ab6d2fee25e7c9e419cffb54d27a99"
server_key_iv: "88e543fd9b733bb82d79ee9257014cad"
server_key_version: "encryptionkey@0"
I20260805 22:44:52.607517 11935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:52.620136 11935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:52.622658 11935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:52.622848 11935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:52.627193 11935 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:34445
I20260805 22:44:52.627279 12032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:34445 every 8 connection(s)
I20260805 22:44:52.627642 11935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:44:52.628583 12033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:52.631049 12033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d: Bootstrap starting.
I20260805 22:44:52.631529 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 11935
I20260805 22:44:52.631625 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:44:52.631739 12033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:52.631860 24341 external_mini_cluster.cc:1428] Setting key cf814705c40fcde3ce33e5d17ef850b3
I20260805 22:44:52.632138 12033 log.cc:824] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:52.632990 12033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d: No bootstrap required, opened a new log
I20260805 22:44:52.634909 12033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06be464580844af6b5550f7b196a956d" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34445 } }
I20260805 22:44:52.635072 12033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:52.635121 12033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06be464580844af6b5550f7b196a956d, State: Initialized, Role: FOLLOWER
I20260805 22:44:52.635262 12033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06be464580844af6b5550f7b196a956d" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34445 } }
I20260805 22:44:52.635366 12033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:44:52.635411 12033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:44:52.635459 12033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:52.635995 12033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06be464580844af6b5550f7b196a956d" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34445 } }
I20260805 22:44:52.636121 12033 leader_election.cc:302] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06be464580844af6b5550f7b196a956d; no voters:
I20260805 22:44:52.636297 12033 leader_election.cc:288] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:44:52.636343 12039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:44:52.636554 12039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [term 1 LEADER]: Becoming Leader. State: Replica: 06be464580844af6b5550f7b196a956d, State: Running, Role: LEADER
I20260805 22:44:52.636597 12033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:44:52.636667 12039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06be464580844af6b5550f7b196a956d" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34445 } }
I20260805 22:44:52.637070 12039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06be464580844af6b5550f7b196a956d. Latest consensus state: current_term: 1 leader_uuid: "06be464580844af6b5550f7b196a956d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06be464580844af6b5550f7b196a956d" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34445 } } }
I20260805 22:44:52.637151 12039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:52.637315 12040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06be464580844af6b5550f7b196a956d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06be464580844af6b5550f7b196a956d" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 34445 } } }
I20260805 22:44:52.637373 12040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:52.637720 12046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:44:52.638182 12046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 05 22:44:52 dist-test-slave-g170 krb5kdc[11920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:52.639946 12046 catalog_manager.cc:1347] Generated new cluster ID: b20fc093d5d24c8c884d94343e356387
I20260805 22:44:52.639995 12046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:44:52.643345 12038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:52.632433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56312 (local address 127.23.197.126:34445)
0805 22:44:52.632630 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:52.632640 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:52.632676 (+ 36us) server_negotiation.cc:407] Connection header received
0805 22:44:52.633213 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:52.633220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:52.633269 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:52.633421 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:52.634103 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:52.634966 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:52.635883 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:52.636459 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:52.636899 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:52.639661 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:52.639665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:52.639665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:52.639677 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:52.641707 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:52.642127 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:52.642130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:52.642131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:52.642235 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:52.642459 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:52.642459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:52.642460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:52.642737 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:52.642878 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:52.643011 (+ 133us) server_negotiation.cc:299] Negotiation successful
0805 22:44:52.643137 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260805 22:44:52.644595 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:44:52.655933 12046 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:44:52.656548 12046 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:44:52.660352 12046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06be464580844af6b5550f7b196a956d: Generated new TSK 0
I20260805 22:44:52.660547 12046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:44:52.699357 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 11935
2026-08-05T22:44:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1564 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:44:52 dist-test-slave-g170 krb5kdc[12079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:44:52 dist-test-slave-g170 krb5kdc[12079](info): set up 2 sockets
Aug 05 22:44:52 dist-test-slave-g170 krb5kdc[12079](info): commencing operation
krb5kdc: starting...
W20260805 22:44:53.972903 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:44:54 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:44:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:44:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:44:54.108075 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41091
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:41091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:54.193558 12112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:54.193821 12112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:54.193869 12112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:54.195080 12112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:44:54.195154 12112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:54.195178 12112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:54.195199 12112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:44:54.195228 12112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:44:54.196806 12112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:41091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:54.197080 12112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:54.197314 12112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:54.202271 12123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:54.202262 12128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:54.202265 12122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:54.202530 12112 server_base.cc:1034] running on GCE node
I20260805 22:44:54.202777 12112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:54.203033 12112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:54.204206 12112 hybrid_clock.cc:648] HybridClock initialized: now 1785969894204183 us; error 44 us; skew 500 ppm
Aug 05 22:44:54 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969894, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:54.233793 12112 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:44:54.234500 12112 webserver.cc:460] Webserver started at http://127.23.197.126:41437/ using document root <none> and password file <none>
I20260805 22:44:54.234764 12112 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:54.234848 12112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:54.235008 12112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:44:54.236169 12112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "083b54d10b394efda4e36d8b7181292e"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "fa1f8540e7a1c94232b91fa1cd01d0a1"
server_key_iv: "e10c2ed0353f04b3b5dcc60e6fa69c55"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.236894 12112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "083b54d10b394efda4e36d8b7181292e"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "fa1f8540e7a1c94232b91fa1cd01d0a1"
server_key_iv: "e10c2ed0353f04b3b5dcc60e6fa69c55"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.238757 12112 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260805 22:44:54.239691 12138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.239969 12112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:54.240116 12112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "083b54d10b394efda4e36d8b7181292e"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "fa1f8540e7a1c94232b91fa1cd01d0a1"
server_key_iv: "e10c2ed0353f04b3b5dcc60e6fa69c55"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.240204 12112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:54.264578 12112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:54.267151 12112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:54.267330 12112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:54.272840 12112 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:41091
I20260805 22:44:54.272900 12222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:41091 every 8 connection(s)
I20260805 22:44:54.273360 12112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:44:54.274684 12225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:54.277192 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 12112
I20260805 22:44:54.277388 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:44:54.277704 24341 external_mini_cluster.cc:1428] Setting key d035af6acd8be3681893358be72bfa8b
I20260805 22:44:54.278959 12225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Bootstrap starting.
I20260805 22:44:54.279807 12225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:54.280369 12225 log.cc:824] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:54.281299 12225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: No bootstrap required, opened a new log
I20260805 22:44:54.284240 12225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:54.284475 12225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:54.284533 12225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083b54d10b394efda4e36d8b7181292e, State: Initialized, Role: FOLLOWER
I20260805 22:44:54.284791 12225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:54.284945 12225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:44:54.284996 12225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:44:54.285068 12225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:54.285907 12225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:54.286041 12225 leader_election.cc:302] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 083b54d10b394efda4e36d8b7181292e; no voters:
I20260805 22:44:54.286279 12225 leader_election.cc:288] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:44:54.286305 12231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:44:54.286546 12231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 1 LEADER]: Becoming Leader. State: Replica: 083b54d10b394efda4e36d8b7181292e, State: Running, Role: LEADER
I20260805 22:44:54.286631 12225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:44:54.286752 12231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:54.287299 12233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "083b54d10b394efda4e36d8b7181292e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } } }
I20260805 22:44:54.287407 12233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:54.287639 12232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 083b54d10b394efda4e36d8b7181292e. Latest consensus state: current_term: 1 leader_uuid: "083b54d10b394efda4e36d8b7181292e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } } }
I20260805 22:44:54.287724 12232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:54.288571 12243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:44:54.289271 12243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 05 22:44:54 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:54.291108 12243 catalog_manager.cc:1347] Generated new cluster ID: 026cfd4ad6da4235a388815b879ea555
I20260805 22:44:54.291213 12243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:44:54.294659 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.278405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.23.197.126:41091)
0805 22:44:54.280364 (+ 1959us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.280374 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.280411 (+ 37us) server_negotiation.cc:407] Connection header received
0805 22:44:54.281053 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.281062 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.281140 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.281293 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:54.282244 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.283908 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.285240 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.285946 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.286444 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.290665 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:54.290669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:54.290670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:54.290683 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:54.292819 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.293316 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.293320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.293320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.293445 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.293688 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.293689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.293689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.293999 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:54.294178 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.294297 (+ 119us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.294443 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1880,"thread_start_us":92,"threads_started":1}
I20260805 22:44:54.304205 12243 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:44:54.305037 12243 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:44:54.313053 12243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Generated new TSK 0
I20260805 22:44:54.313441 12243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:44:54.347990 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41091
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:54.436195 12258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:54.436465 12258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:54.436511 12258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:54.437714 12258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:54.437783 12258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:54.437829 12258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:44:54.439343 12258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:54.439635 12258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:54.439905 12258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:54.445335 12270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:54.445329 12258 server_base.cc:1034] running on GCE node
W20260805 22:44:54.445329 12269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:54.445343 12275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:54.445837 12258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:54.446062 12258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:54.447242 12258 hybrid_clock.cc:648] HybridClock initialized: now 1785969894447232 us; error 36 us; skew 500 ppm
Aug 05 22:44:54 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969894, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:54.450053 12258 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:44:54.450613 12258 webserver.cc:460] Webserver started at http://127.23.197.65:38199/ using document root <none> and password file <none>
I20260805 22:44:54.450866 12258 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:54.450940 12258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:54.451098 12258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:44:54.452235 12258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "bc1996c8f1f841d5a0c3709f7fdc834d"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "149cc5de35c2f745c12f06edb5fa7e1c"
server_key_iv: "1bfb389b57a5692345cd73dc3ec34b63"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.452656 12258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "bc1996c8f1f841d5a0c3709f7fdc834d"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "149cc5de35c2f745c12f06edb5fa7e1c"
server_key_iv: "1bfb389b57a5692345cd73dc3ec34b63"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.454200 12258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:44:54.455113 12282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.455366 12258 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:54.455483 12258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "bc1996c8f1f841d5a0c3709f7fdc834d"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "149cc5de35c2f745c12f06edb5fa7e1c"
server_key_iv: "1bfb389b57a5692345cd73dc3ec34b63"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.455562 12258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:54.474020 12258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:54.476249 12258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:54.476441 12258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:54.476708 12258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:54.477109 12258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:54.477167 12258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.477203 12258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:54.477236 12258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.483948 12258 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:33061
I20260805 22:44:54.483989 12445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:33061 every 8 connection(s)
I20260805 22:44:54.484345 12258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
Aug 05 22:44:54 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969894, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:54.494463 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 12258
I20260805 22:44:54.494570 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:44:54.494783 24341 external_mini_cluster.cc:1428] Setting key 3eb6eff41fe8dd6feb052cc79fd05436
I20260805 22:44:54.495416 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.485129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:48519 (local address 127.23.197.126:41091)
0805 22:44:54.485209 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.485211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.485441 (+ 230us) server_negotiation.cc:407] Connection header received
0805 22:44:54.485995 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.485997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.486009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.486066 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:54.487166 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.487958 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.489121 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.489359 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.489456 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.491751 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:54.491753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:54.491753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:54.491761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:54.493556 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.494080 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.494081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.494082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.494148 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.494457 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.494458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.494458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.494627 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:54.494760 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.495195 (+ 435us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.495326 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 22:44:54.495671 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.485109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:48519)
0805 22:44:54.485304 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:54.485344 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:54.485879 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:54.486154 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:54.486157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:54.486819 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:54.487027 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.487033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.488080 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.488081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.489016 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.489018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.489471 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.489472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.489605 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.490165 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:54.490175 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:54.491638 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:54.493699 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:54.493709 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:54.493711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:54.493976 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:54.494237 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:54.494238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:54.494239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:54.494375 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:54.494764 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:54.494773 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:54.494971 (+ 198us) client_negotiation.cc:769] Sending connection context
0805 22:44:54.495148 (+ 177us) client_negotiation.cc:240] Negotiation successful
0805 22:44:54.495379 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260805 22:44:54.496261 12446 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41091
I20260805 22:44:54.496384 12446 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:54.496603 12446 heartbeater.cc:511] Master 127.23.197.126:41091 requested a full tablet report, sending...
I20260805 22:44:54.497172 12151 ts_manager.cc:194] Registered new tserver with Master: bc1996c8f1f841d5a0c3709f7fdc834d (127.23.197.65:33061)
I20260805 22:44:54.498260 12151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:48519
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:44:54.524803 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41091
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:54.609328 12453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:54.609555 12453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:54.609580 12453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:54.610747 12453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:54.610786 12453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:54.610812 12453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:44:54.612210 12453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:54.612414 12453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:54.612591 12453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:54.617797 12469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:54.617805 12464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:54.617807 12466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:54.618095 12453 server_base.cc:1034] running on GCE node
I20260805 22:44:54.618350 12453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:54.618602 12453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:54.619807 12453 hybrid_clock.cc:648] HybridClock initialized: now 1785969894619796 us; error 44 us; skew 500 ppm
Aug 05 22:44:54 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969894, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:54.622967 12453 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:44:54.623476 12453 webserver.cc:460] Webserver started at http://127.23.197.66:38457/ using document root <none> and password file <none>
I20260805 22:44:54.623718 12453 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:54.623764 12453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:54.623867 12453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:44:54.625106 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "61c2c7417b27442686e8cbe4ad7ab785"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "3710b77c33adf0a7f5642c3896506f9e"
server_key_iv: "506d6d915d882a0edc8cab5e167a3c39"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.625465 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "61c2c7417b27442686e8cbe4ad7ab785"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "3710b77c33adf0a7f5642c3896506f9e"
server_key_iv: "506d6d915d882a0edc8cab5e167a3c39"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.626890 12453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:44:54.627713 12479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.627938 12453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:54.628073 12453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "61c2c7417b27442686e8cbe4ad7ab785"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "3710b77c33adf0a7f5642c3896506f9e"
server_key_iv: "506d6d915d882a0edc8cab5e167a3c39"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.628170 12453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:54.641729 12453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:54.643903 12453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:54.644097 12453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:54.644387 12453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:54.644747 12453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:54.644915 12453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.644964 12453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:54.645011 12453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.651937 12453 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:40115
I20260805 22:44:54.651975 12648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:40115 every 8 connection(s)
I20260805 22:44:54.652374 12453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
Aug 05 22:44:54 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969894, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:54.661293 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 12453
I20260805 22:44:54.661441 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:44:54.661697 24341 external_mini_cluster.cc:1428] Setting key 1d3a9d561987da8ddf4e0612bc7a45b4
I20260805 22:44:54.663069 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.653259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:43195 (local address 127.23.197.126:41091)
0805 22:44:54.653329 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.653336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.653555 (+ 219us) server_negotiation.cc:407] Connection header received
0805 22:44:54.654108 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.654109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.654123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.654185 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:54.655163 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.655671 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.657021 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.657253 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.657343 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.659653 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:54.659654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:54.659655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:54.659663 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:54.661260 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.661811 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.661813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.661813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.661884 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.662163 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.662164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.662165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.662289 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:54.662360 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.662927 (+ 567us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.663006 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 22:44:54.663240 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.653244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:43195)
0805 22:44:54.653424 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:54.653462 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:54.654000 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:54.654266 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:54.654270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:54.654863 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:54.655061 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.655066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.655832 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.655834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.656909 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.656912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.657348 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.657349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.657491 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.658068 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:54.658077 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:54.659541 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:54.661360 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:54.661368 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:54.661370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:54.661728 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:54.661957 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:54.661958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:54.661958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:54.662106 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:54.662364 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:54.662371 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:54.662512 (+ 141us) client_negotiation.cc:769] Sending connection context
0805 22:44:54.662718 (+ 206us) client_negotiation.cc:240] Negotiation successful
0805 22:44:54.662949 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260805 22:44:54.663811 12650 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41091
I20260805 22:44:54.663916 12650 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:54.664116 12650 heartbeater.cc:511] Master 127.23.197.126:41091 requested a full tablet report, sending...
I20260805 22:44:54.664564 12151 ts_manager.cc:194] Registered new tserver with Master: 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115)
I20260805 22:44:54.665051 12151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:43195
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:44:54.692268 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41091
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:54.778467 12656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:54.778700 12656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:54.778739 12656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:54.779870 12656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:54.779932 12656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:54.779976 12656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:44:54.781584 12656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:41091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:54.781836 12656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:54.782070 12656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:54.787443 12664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:54.787456 12656 server_base.cc:1034] running on GCE node
W20260805 22:44:54.787596 12663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:54.787503 12669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:54.787978 12656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:54.788239 12656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:54.789464 12656 hybrid_clock.cc:648] HybridClock initialized: now 1785969894789453 us; error 33 us; skew 500 ppm
Aug 05 22:44:54 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969894, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:54.792346 12656 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:44:54.792985 12656 webserver.cc:460] Webserver started at http://127.23.197.67:44369/ using document root <none> and password file <none>
I20260805 22:44:54.793249 12656 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:54.793320 12656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:54.793476 12656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:44:54.794605 12656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "a6923f848bd64c2ea3a94d2b2aaade02"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "c48ee41b83e63d165b0765d894a5ef59"
server_key_iv: "8e6609427e0305d2048b58e39f1e0212"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.795006 12656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "a6923f848bd64c2ea3a94d2b2aaade02"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "c48ee41b83e63d165b0765d894a5ef59"
server_key_iv: "8e6609427e0305d2048b58e39f1e0212"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.796464 12656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:44:54.797439 12676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.797672 12656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:54.797778 12656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "a6923f848bd64c2ea3a94d2b2aaade02"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "c48ee41b83e63d165b0765d894a5ef59"
server_key_iv: "8e6609427e0305d2048b58e39f1e0212"
server_key_version: "encryptionkey@0"
I20260805 22:44:54.797863 12656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:54.838537 12656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:54.841403 12656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:54.841600 12656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:54.841918 12656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:54.842312 12656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:54.842355 12656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.842383 12656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:54.842399 12656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:54.850057 12656 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:40859
I20260805 22:44:54.850207 12849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:40859 every 8 connection(s)
I20260805 22:44:54.850556 12656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
Aug 05 22:44:54 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969894, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:54.859097 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 12656
I20260805 22:44:54.859256 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:44:54.859467 24341 external_mini_cluster.cc:1428] Setting key eea4ce31a9cc173c712d4ff2be8fc573
I20260805 22:44:54.861652 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.851503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:49429 (local address 127.23.197.126:41091)
0805 22:44:54.851600 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.851602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.851850 (+ 248us) server_negotiation.cc:407] Connection header received
0805 22:44:54.852359 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.852361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.852374 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.852445 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:54.853527 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.854061 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.855114 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.855317 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.855386 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.857808 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:54.857810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:54.857810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:54.857818 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:54.859967 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.860517 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.860518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.860519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.860587 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.860940 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.860941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.860942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.861079 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:54.861159 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.861464 (+ 305us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.861555 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 22:44:54.861872 12855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.851473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:49429)
0805 22:44:54.851684 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:54.851723 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:54.852242 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:54.852502 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:54.852505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:54.853184 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:54.853410 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.853415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.854157 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.854158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.855011 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.855014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.855403 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.855404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.855506 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.856033 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:54.856042 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:54.857691 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:54.860041 (+ 2350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:54.860052 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:54.860055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:54.860428 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:54.860665 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:54.860678 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:54.860679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:54.860866 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:54.861148 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:54.861157 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:54.861318 (+ 161us) client_negotiation.cc:769] Sending connection context
0805 22:44:54.861428 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 22:44:54.861617 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":128,"threads_started":1}
I20260805 22:44:54.862424 12850 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41091
I20260805 22:44:54.862533 12850 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:54.862721 12850 heartbeater.cc:511] Master 127.23.197.126:41091 requested a full tablet report, sending...
I20260805 22:44:54.863214 12151 ts_manager.cc:194] Registered new tserver with Master: a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859)
I20260805 22:44:54.863592 12151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:49429
I20260805 22:44:54.871781 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:44:54.879690 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.873249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.23.197.126:41091)
0805 22:44:54.873345 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.873347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.873408 (+ 61us) server_negotiation.cc:407] Connection header received
0805 22:44:54.873455 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.873457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.873467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.873519 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:54.874126 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.874643 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.875450 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.875638 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.875698 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.877066 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:54.877067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:54.877068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:54.877075 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:54.878443 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.878794 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.878796 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.878796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.878865 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:54.879141 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:54.879143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:54.879143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:54.879322 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:54.879421 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.879510 (+ 89us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.879576 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260805 22:44:54.881683 12151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:44:54.883356 12151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:44:54.890494 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.885994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:40859 (local address 127.0.0.1:36760)
0805 22:44:54.886162 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:54.886206 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:54.886280 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:54.886641 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:54.886644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:54.886646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:54.886719 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.886721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.888108 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.888109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.889286 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.889288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.890172 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.890173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.890252 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.890267 (+ 15us) client_negotiation.cc:769] Sending connection context
0805 22:44:54.890342 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 22:44:54.890402 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":62,"spinlock_wait_cycles":768,"thread_start_us":98,"threads_started":1}
I20260805 22:44:54.890514 12872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.886107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36760 (local address 127.23.197.67:40859)
0805 22:44:54.886382 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.886386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.886402 (+ 16us) server_negotiation.cc:407] Connection header received
0805 22:44:54.886434 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.886436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.886448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.886542 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:54.886810 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.887987 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.889379 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.890067 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.890156 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.890214 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.890384 (+ 170us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.890429 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":124,"threads_started":1}
I20260805 22:44:54.890494 12875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.885995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40115 (local address 127.0.0.1:51538)
0805 22:44:54.886577 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:54.886589 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:54.886650 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:54.886954 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:54.886955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:54.886957 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:54.886998 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.887000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.888285 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.888286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.889517 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.889519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.890276 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.890276 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.890337 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.890341 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:44:54.890386 (+ 45us) client_negotiation.cc:240] Negotiation successful
0805 22:44:54.890415 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":3,"thread_start_us":66,"threads_started":1}
I20260805 22:44:54.890856 12877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.886394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.23.197.65:33061)
0805 22:44:54.886600 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.886604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.886715 (+ 111us) server_negotiation.cc:407] Connection header received
0805 22:44:54.886763 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.886766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.886783 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.886853 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:54.887053 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.887991 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.889606 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.890299 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.890384 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.890437 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.890727 (+ 290us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.890769 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":96,"threads_started":1}
I20260805 22:44:54.891027 12870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.886223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33061 (local address 127.0.0.1:38054)
0805 22:44:54.886577 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:54.886589 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:54.886650 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:54.886933 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:54.886934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:54.886936 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:54.886990 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.886992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.888087 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.888088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.889507 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.889510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.890473 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.890474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.890538 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.890543 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:44:54.890631 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 22:44:54.890678 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":108,"thread_start_us":88,"threads_started":1}
I20260805 22:44:54.891355 12878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.886405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51538 (local address 127.23.197.66:40115)
0805 22:44:54.886691 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.886697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.886706 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:44:54.886741 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.886744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.886800 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.886878 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:54.887074 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.888190 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.889623 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.890094 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.890184 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.890243 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.890537 (+ 294us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.891262 (+ 725us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":89,"threads_started":1}
I20260805 22:44:54.891306 12751 tablet_service.cc:1467] Processing CreateTablet for tablet fa94ad25408043cea5173d7578521047 (DEFAULT_TABLE table=test-table [id=a82d7c718ec14ee7b420e53e602a7cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:44:54.891709 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa94ad25408043cea5173d7578521047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:54.891637 12347 tablet_service.cc:1467] Processing CreateTablet for tablet fa94ad25408043cea5173d7578521047 (DEFAULT_TABLE table=test-table [id=a82d7c718ec14ee7b420e53e602a7cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:44:54.891975 12347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa94ad25408043cea5173d7578521047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:54.892151 12552 tablet_service.cc:1467] Processing CreateTablet for tablet fa94ad25408043cea5173d7578521047 (DEFAULT_TABLE table=test-table [id=a82d7c718ec14ee7b420e53e602a7cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:44:54.892480 12552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa94ad25408043cea5173d7578521047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:54.894901 12879 tablet_bootstrap.cc:492] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: Bootstrap starting.
I20260805 22:44:54.895507 12880 tablet_bootstrap.cc:492] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Bootstrap starting.
I20260805 22:44:54.895939 12881 tablet_bootstrap.cc:492] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: Bootstrap starting.
I20260805 22:44:54.896003 12879 tablet_bootstrap.cc:654] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:54.896301 12880 tablet_bootstrap.cc:654] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:54.896567 12881 tablet_bootstrap.cc:654] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:54.896724 12879 log.cc:824] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:54.897012 12880 log.cc:824] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:54.897250 12881 log.cc:824] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:54.897717 12879 tablet_bootstrap.cc:492] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: No bootstrap required, opened a new log
I20260805 22:44:54.897805 12879 ts_tablet_manager.cc:1397] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260805 22:44:54.898000 12881 tablet_bootstrap.cc:492] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: No bootstrap required, opened a new log
I20260805 22:44:54.898077 12881 ts_tablet_manager.cc:1397] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:44:54.899622 12881 raft_consensus.cc:348] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:54.899566 12879 raft_consensus.cc:348] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:54.899763 12879 raft_consensus.cc:374] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:54.899765 12881 raft_consensus.cc:374] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:54.899816 12881 raft_consensus.cc:729] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c2c7417b27442686e8cbe4ad7ab785, State: Initialized, Role: FOLLOWER
I20260805 22:44:54.899860 12879 raft_consensus.cc:729] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6923f848bd64c2ea3a94d2b2aaade02, State: Initialized, Role: FOLLOWER
I20260805 22:44:54.899946 12881 consensus_queue.cc:260] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:54.900045 12879 consensus_queue.cc:260] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:54.900312 12881 ts_tablet_manager.cc:1428] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:44:54.900314 12650 heartbeater.cc:503] Master 127.23.197.126:41091 was elected leader, sending a full tablet report...
I20260805 22:44:54.900420 12850 heartbeater.cc:503] Master 127.23.197.126:41091 was elected leader, sending a full tablet report...
I20260805 22:44:54.901041 12880 tablet_bootstrap.cc:492] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: No bootstrap required, opened a new log
I20260805 22:44:54.901154 12880 ts_tablet_manager.cc:1397] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260805 22:44:54.902662 12880 raft_consensus.cc:348] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:54.902827 12880 raft_consensus.cc:374] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:54.902886 12880 raft_consensus.cc:729] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc1996c8f1f841d5a0c3709f7fdc834d, State: Initialized, Role: FOLLOWER
W20260805 22:44:54.902984 12651 tablet.cc:2369] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:44:54.903014 12880 consensus_queue.cc:260] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:54.903317 12880 ts_tablet_manager.cc:1428] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:44:54.903422 12446 heartbeater.cc:503] Master 127.23.197.126:41091 was elected leader, sending a full tablet report...
I20260805 22:44:54.903465 12879 ts_tablet_manager.cc:1428] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260805 22:44:54.967619 12885 raft_consensus.cc:482] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:44:54.967772 12885 raft_consensus.cc:504] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:54.968225 12885 leader_election.cc:288] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859), bc1996c8f1f841d5a0c3709f7fdc834d (127.23.197.65:33061)
I20260805 22:44:54.971765 12652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.968472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:40859 (local address 127.23.197.66:42827)
0805 22:44:54.968556 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:54.968572 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:54.968642 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:54.968945 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:54.968947 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:54.968951 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:54.969054 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.969056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.969862 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.969863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.970888 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.970890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.971503 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.971504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.971603 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.971607 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:44:54.971666 (+ 59us) client_negotiation.cc:240] Negotiation successful
0805 22:44:54.971704 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":49}
I20260805 22:44:54.971833 12872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.968608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:42827 (local address 127.23.197.67:40859)
0805 22:44:54.968753 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.968755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.968799 (+ 44us) server_negotiation.cc:407] Connection header received
0805 22:44:54.968845 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.968846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.968856 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.968925 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:54.969175 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.969751 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.970999 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.971419 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.971480 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.971488 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.971706 (+ 218us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.971760 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260805 22:44:54.972087 12877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.968743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:55209 (local address 127.23.197.65:33061)
0805 22:44:54.968882 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.968884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.968920 (+ 36us) server_negotiation.cc:407] Connection header received
0805 22:44:54.968996 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.968998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.969008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.969078 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:54.969379 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.969990 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.971054 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.971493 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.971575 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.971585 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.971969 (+ 384us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.972023 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260805 22:44:54.972318 12786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa94ad25408043cea5173d7578521047" candidate_uuid: "61c2c7417b27442686e8cbe4ad7ab785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" is_pre_election: true
I20260805 22:44:54.972396 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.968627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33061 (local address 127.23.197.66:55209)
0805 22:44:54.968870 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:54.968881 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:54.968932 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:54.969173 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:54.969174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:54.969177 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:54.969255 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.969256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.970107 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.970108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.970920 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:54.970923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.971702 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.971703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:54.971849 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.971855 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 22:44:54.972198 (+ 343us) client_negotiation.cc:240] Negotiation successful
0805 22:44:54.972242 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":136,"threads_started":1}
I20260805 22:44:54.972546 12786 raft_consensus.cc:2454] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61c2c7417b27442686e8cbe4ad7ab785 in term 0.
I20260805 22:44:54.972872 12480 leader_election.cc:302] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61c2c7417b27442686e8cbe4ad7ab785, a6923f848bd64c2ea3a94d2b2aaade02; no voters:
I20260805 22:44:54.972802 12380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa94ad25408043cea5173d7578521047" candidate_uuid: "61c2c7417b27442686e8cbe4ad7ab785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" is_pre_election: true
I20260805 22:44:54.973057 12885 raft_consensus.cc:2781] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:44:54.973094 12380 raft_consensus.cc:2454] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61c2c7417b27442686e8cbe4ad7ab785 in term 0.
I20260805 22:44:54.973114 12885 raft_consensus.cc:482] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:44:54.973155 12885 raft_consensus.cc:3037] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:54.974004 12885 raft_consensus.cc:504] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:54.974161 12885 leader_election.cc:288] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [CANDIDATE]: Term 1 election: Requested vote from peers a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859), bc1996c8f1f841d5a0c3709f7fdc834d (127.23.197.65:33061)
I20260805 22:44:54.974347 12786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa94ad25408043cea5173d7578521047" candidate_uuid: "61c2c7417b27442686e8cbe4ad7ab785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6923f848bd64c2ea3a94d2b2aaade02"
I20260805 22:44:54.974385 12380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa94ad25408043cea5173d7578521047" candidate_uuid: "61c2c7417b27442686e8cbe4ad7ab785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d"
I20260805 22:44:54.974442 12786 raft_consensus.cc:3037] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:54.974495 12380 raft_consensus.cc:3037] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:54.975230 12786 raft_consensus.cc:2454] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61c2c7417b27442686e8cbe4ad7ab785 in term 1.
I20260805 22:44:54.975373 12380 raft_consensus.cc:2454] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61c2c7417b27442686e8cbe4ad7ab785 in term 1.
I20260805 22:44:54.975420 12480 leader_election.cc:302] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61c2c7417b27442686e8cbe4ad7ab785, a6923f848bd64c2ea3a94d2b2aaade02; no voters:
I20260805 22:44:54.975508 12885 raft_consensus.cc:2781] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:44:54.975754 12885 raft_consensus.cc:686] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 1 LEADER]: Becoming Leader. State: Replica: 61c2c7417b27442686e8cbe4ad7ab785, State: Running, Role: LEADER
I20260805 22:44:54.975879 12885 consensus_queue.cc:237] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:54.976955 12162 catalog_manager.cc:5495] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "61c2c7417b27442686e8cbe4ad7ab785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } health_report { overall_health: HEALTHY } } }
W20260805 22:44:54.985072 12447 tablet.cc:2369] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:44:54.986423 12878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:54.983779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51544 (local address 127.23.197.66:40115)
0805 22:44:54.983852 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:54.983854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:54.983898 (+ 44us) server_negotiation.cc:407] Connection header received
0805 22:44:54.983928 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:54.983929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:54.983946 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:54.984006 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:44:54.984232 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.984719 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.985693 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:54.985890 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:54.985954 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:54.986103 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:44:54.986220 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:44:54.986288 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:54.986329 (+ 41us) server_negotiation.cc:299] Negotiation successful
0805 22:44:54.986348 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 22:44:54.988499 12380 raft_consensus.cc:1261] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Refusing update from remote peer 61c2c7417b27442686e8cbe4ad7ab785: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:44:54.988498 12786 raft_consensus.cc:1261] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [term 1 FOLLOWER]: Refusing update from remote peer 61c2c7417b27442686e8cbe4ad7ab785: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:44:54.988966 12892 consensus_queue.cc:1035] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:44:54.989122 12885 consensus_queue.cc:1035] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:44:54.991820 12901 mvcc.cc:205] Tried to move back new op lower bound from 7315332689870102528 to 7315332689821798400. Current Snapshot: MvccSnapshot[applied={T|T < 7315332689870102528}]
I20260805 22:44:54.992491 12902 mvcc.cc:205] Tried to move back new op lower bound from 7315332689870102528 to 7315332689821798400. Current Snapshot: MvccSnapshot[applied={T|T < 7315332689870102528 or (T in {7315332689870102528})}]
I20260805 22:44:54.993136 12904 mvcc.cc:205] Tried to move back new op lower bound from 7315332689870102528 to 7315332689821798400. Current Snapshot: MvccSnapshot[applied={T|T < 7315332689870102528}]
I20260805 22:44:54.994875 12161 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:44:54.994984 12161 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:44:54.995867 12161 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Sending DeleteTablet for 3 replicas of tablet fa94ad25408043cea5173d7578521047
I20260805 22:44:54.996227 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 12258
I20260805 22:44:54.996304 12347 tablet_service.cc:1514] Processing DeleteTablet for tablet fa94ad25408043cea5173d7578521047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:44:54 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:38054
I20260805 22:44:54.996819 12751 tablet_service.cc:1514] Processing DeleteTablet for tablet fa94ad25408043cea5173d7578521047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:44:54 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:36760
I20260805 22:44:54.998620 12916 tablet_replica.cc:330] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: stopping tablet replica
I20260805 22:44:54.998629 12552 tablet_service.cc:1514] Processing DeleteTablet for tablet fa94ad25408043cea5173d7578521047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:44:54 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:51538
I20260805 22:44:54.998734 12916 raft_consensus.cc:2229] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:44:54.998842 12916 raft_consensus.cc:2258] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:44:55.000728 12916 ts_tablet_manager.cc:1905] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:44:55.003155 12916 ts_tablet_manager.cc:1918] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:44:55.003240 12916 log.cc:1196] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/fa94ad25408043cea5173d7578521047
I20260805 22:44:55.003494 12916 ts_tablet_manager.cc:1939] T fa94ad25408043cea5173d7578521047 P a6923f848bd64c2ea3a94d2b2aaade02: Deleting consensus metadata
I20260805 22:44:55.004627 12139 catalog_manager.cc:4841] TS a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859): tablet fa94ad25408043cea5173d7578521047 (table test-table [id=a82d7c718ec14ee7b420e53e602a7cfb]) successfully deleted
I20260805 22:44:55.008955 12919 tablet_replica.cc:330] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: stopping tablet replica
I20260805 22:44:55.009155 12919 raft_consensus.cc:2229] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:44:55.009344 12919 raft_consensus.cc:2258] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:44:55.009895 12919 ts_tablet_manager.cc:1905] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:44:55.011525 12919 ts_tablet_manager.cc:1918] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:44:55.011623 12919 log.cc:1196] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/fa94ad25408043cea5173d7578521047
I20260805 22:44:55.011940 12919 ts_tablet_manager.cc:1939] T fa94ad25408043cea5173d7578521047 P 61c2c7417b27442686e8cbe4ad7ab785: Deleting consensus metadata
I20260805 22:44:55.012480 12142 catalog_manager.cc:4841] TS 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115): tablet fa94ad25408043cea5173d7578521047 (table test-table [id=a82d7c718ec14ee7b420e53e602a7cfb]) successfully deleted
W20260805 22:44:55.014017 12139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:33061 (error 108)
I20260805 22:44:55.014259 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 12453
I20260805 22:44:55.025383 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.025197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33061 (local address 127.0.0.1:38060)
0805 22:44:55.025283 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.025343 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260805 22:44:55.025557 12139 catalog_manager.cc:4584] TS bc1996c8f1f841d5a0c3709f7fdc834d (127.23.197.65:33061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa94ad25408043cea5173d7578521047: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33061: connect: Connection refused (error 111)
I20260805 22:44:55.026070 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 12656
I20260805 22:44:55.051044 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 12112
I20260805 22:44:55.061293 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41091
--webserver_interface=127.23.197.126
--webserver_port=41437
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:41091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:55.172649 12922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:55.172999 12922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:55.173043 12922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:55.174407 12922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:44:55.174480 12922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:55.174510 12922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:55.174544 12922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:44:55.174575 12922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:44:55.176307 12922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:41091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=41437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:55.176631 12922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:55.176939 12922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:55.187264 12933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.187234 12937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.187258 12932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:55.187662 12922 server_base.cc:1034] running on GCE node
I20260805 22:44:55.187959 12922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:55.188297 12922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:55.189447 12922 hybrid_clock.cc:648] HybridClock initialized: now 1785969895189426 us; error 39 us; skew 500 ppm
Aug 05 22:44:55 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969895, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:55.192114 12922 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:44:55.192521 12922 webserver.cc:460] Webserver started at http://127.23.197.126:41437/ using document root <none> and password file <none>
I20260805 22:44:55.192746 12922 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:55.192863 12922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:55.194281 12922 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.194993 12945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:55.195174 12922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.195294 12922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "083b54d10b394efda4e36d8b7181292e"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "fa1f8540e7a1c94232b91fa1cd01d0a1"
server_key_iv: "e10c2ed0353f04b3b5dcc60e6fa69c55"
server_key_version: "encryptionkey@0"
I20260805 22:44:55.195708 12922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:55.203893 12922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:55.206370 12922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:55.206543 12922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:55.211105 12922 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:41091
I20260805 22:44:55.211154 13037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:41091 every 8 connection(s)
I20260805 22:44:55.211632 12922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:44:55.213668 13038 sys_catalog.cc:263] Verifying existing consensus state
I20260805 22:44:55.214427 13038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Bootstrap starting.
I20260805 22:44:55.218907 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 12922
I20260805 22:44:55.219255 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:33061
--local_ip_for_outbound_sockets=127.23.197.65
--tserver_master_addrs=127.23.197.126:41091
--webserver_port=38199
--webserver_interface=127.23.197.65
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:44:55.224201 13038 log.cc:824] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:55.226240 13038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:44:55.226578 13038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Bootstrap complete.
I20260805 22:44:55.229137 13038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:55.229372 13038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083b54d10b394efda4e36d8b7181292e, State: Initialized, Role: FOLLOWER
I20260805 22:44:55.229545 13038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:55.229616 13038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:44:55.229686 13038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:44:55.229735 13038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 1 FOLLOWER]: Advancing to term 2
I20260805 22:44:55.230803 13038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:55.230890 13038 leader_election.cc:302] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 083b54d10b394efda4e36d8b7181292e; no voters:
I20260805 22:44:55.231117 13038 leader_election.cc:288] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 22:44:55.231154 13045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 2 FOLLOWER]: Leader election won for term 2
I20260805 22:44:55.231261 13045 raft_consensus.cc:686] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 2 LEADER]: Becoming Leader. State: Replica: 083b54d10b394efda4e36d8b7181292e, State: Running, Role: LEADER
I20260805 22:44:55.231397 13038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:44:55.231374 13045 consensus_queue.cc:237] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:55.232015 13047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 083b54d10b394efda4e36d8b7181292e. Latest consensus state: current_term: 2 leader_uuid: "083b54d10b394efda4e36d8b7181292e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } } }
I20260805 22:44:55.232048 13046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "083b54d10b394efda4e36d8b7181292e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } } }
I20260805 22:44:55.232218 13047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:55.232234 13046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:55.232483 13061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:44:55.233657 13061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:44:55.233835 13061 catalog_manager.cc:1259] Loaded cluster ID: 026cfd4ad6da4235a388815b879ea555
I20260805 22:44:55.234078 13061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:44:55.234292 13061 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260805 22:44:55.237026 13061 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260805 22:44:55.241828 13061 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:44:55.242063 13061 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Loaded TSK: 0
I20260805 22:44:55.242323 13061 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 22:44:55.323709 13043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:55.323956 13043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:55.323997 13043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:55.325320 13043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:55.325383 13043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:55.325428 13043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:44:55.326915 13043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:33061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=38199
--tserver_master_addrs=127.23.197.126:41091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:55.327185 13043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:55.327515 13043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:55.332964 13085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.332973 13086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.332973 13088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:55.333407 13043 server_base.cc:1034] running on GCE node
I20260805 22:44:55.333614 13043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:55.333873 13043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:55.335040 13043 hybrid_clock.cc:648] HybridClock initialized: now 1785969895335027 us; error 33 us; skew 500 ppm
Aug 05 22:44:55 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969895, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:55.337868 13043 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:44:55.338362 13043 webserver.cc:460] Webserver started at http://127.23.197.65:38199/ using document root <none> and password file <none>
I20260805 22:44:55.338598 13043 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:55.338686 13043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:55.340113 13043 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.340898 13100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:55.341068 13043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.341158 13043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "bc1996c8f1f841d5a0c3709f7fdc834d"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "149cc5de35c2f745c12f06edb5fa7e1c"
server_key_iv: "1bfb389b57a5692345cd73dc3ec34b63"
server_key_version: "encryptionkey@0"
I20260805 22:44:55.341503 13043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:55.349570 13043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:55.351830 13043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:55.352092 13043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:55.352332 13043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:55.352821 13110 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 22:44:55.354759 13043 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 22:44:55.354828 13043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260805 22:44:55.354872 13043 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 22:44:55.355512 13043 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 22:44:55.355651 13043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.355643 13110 tablet_bootstrap.cc:492] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Bootstrap starting.
I20260805 22:44:55.365008 13043 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:33061
I20260805 22:44:55.365054 13253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:33061 every 8 connection(s)
I20260805 22:44:55.365494 13043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:44:55.365875 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 13043
I20260805 22:44:55.366151 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:40115
--local_ip_for_outbound_sockets=127.23.197.66
--tserver_master_addrs=127.23.197.126:41091
--webserver_port=38457
--webserver_interface=127.23.197.66
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:44:55.369704 13110 log.cc:824] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:55.372395 13110 tablet_bootstrap.cc:492] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:44:55.372712 13110 tablet_bootstrap.cc:492] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Bootstrap complete.
I20260805 22:44:55.373961 13110 ts_tablet_manager.cc:1397] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.003s
I20260805 22:44:55.375608 13110 raft_consensus.cc:348] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:55.375965 13110 raft_consensus.cc:729] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc1996c8f1f841d5a0c3709f7fdc834d, State: Initialized, Role: FOLLOWER
I20260805 22:44:55.376137 13110 consensus_queue.cc:260] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } }
I20260805 22:44:55.376398 13110 ts_tablet_manager.cc:1428] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
Aug 05 22:44:55 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969895, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:55.389379 13262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.368641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:56463)
0805 22:44:55.376885 (+ 8244us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.376935 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:55.377139 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:55.377649 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:55.377654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:55.378041 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:55.378341 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.378348 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.380097 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.380098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.381454 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.381458 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.382344 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.382346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.382501 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.383357 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:55.383367 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:55.385326 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:55.387514 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:55.387522 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:55.387524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:55.387823 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:55.388100 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:55.388101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:55.388101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:55.388243 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:55.388684 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:55.388690 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:55.388864 (+ 174us) client_negotiation.cc:769] Sending connection context
0805 22:44:55.388967 (+ 103us) client_negotiation.cc:240] Negotiation successful
0805 22:44:55.389148 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8185,"thread_start_us":120,"threads_started":1}
I20260805 22:44:55.389420 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.369026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:56463 (local address 127.23.197.126:41091)
0805 22:44:55.376913 (+ 7887us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.376924 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.377065 (+ 141us) server_negotiation.cc:407] Connection header received
0805 22:44:55.377350 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.377358 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.377425 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.377582 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:55.378484 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.379970 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.381553 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.382200 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.382321 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.385443 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:55.385448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:55.385448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:55.385461 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:55.387420 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.387920 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.387923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.387924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.388031 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.388318 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.388319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.388319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.388606 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:55.388709 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.389073 (+ 364us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.389222 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7831,"thread_start_us":87,"threads_started":1}
I20260805 22:44:55.390825 13254 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41091
I20260805 22:44:55.390945 13254 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:55.391114 13254 heartbeater.cc:511] Master 127.23.197.126:41091 requested a full tablet report, sending...
I20260805 22:44:55.391904 12977 ts_manager.cc:194] Registered new tserver with Master: bc1996c8f1f841d5a0c3709f7fdc834d (127.23.197.65:33061)
I20260805 22:44:55.393086 12977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:56463
I20260805 22:44:55.394035 13254 heartbeater.cc:503] Master 127.23.197.126:41091 was elected leader, sending a full tablet report...
Aug 05 22:44:55 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969895, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:44:55.401386 13270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.392548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33061 (local address 127.0.0.1:38066)
0805 22:44:55.393400 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.393434 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:55.393503 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:55.393701 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:55.393706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:55.394033 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:55.394101 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.394103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.395077 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.395078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.395693 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.395696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.396177 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.396178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.396263 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.396727 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:55.396731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:55.398335 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:55.400198 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:55.400204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:55.400206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:55.400404 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:55.400633 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:55.400634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:55.400635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:55.400699 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:55.401004 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:55.401010 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:55.401063 (+ 53us) client_negotiation.cc:769] Sending connection context
0805 22:44:55.401166 (+ 103us) client_negotiation.cc:240] Negotiation successful
0805 22:44:55.401296 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":104,"threads_started":1}
I20260805 22:44:55.401388 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.392863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38066 (local address 127.23.197.65:33061)
0805 22:44:55.393049 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.393053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.393494 (+ 441us) server_negotiation.cc:407] Connection header received
0805 22:44:55.393588 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.393590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.393605 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.393693 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:55.394196 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.394983 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.395787 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.396090 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.396177 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.398432 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:55.398465 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:55.398466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:55.398474 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:55.400085 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.400494 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.400497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.400497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.400558 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.400785 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.400786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.400786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.400919 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:55.401029 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.401202 (+ 173us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.401289 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260805 22:44:55.401774 13153 tablet_service.cc:1514] Processing DeleteTablet for tablet fa94ad25408043cea5173d7578521047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:44:54 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:38066
I20260805 22:44:55.402290 13272 tablet_replica.cc:330] stopping tablet replica
I20260805 22:44:55.402410 13272 raft_consensus.cc:2229] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:44:55.402480 13272 raft_consensus.cc:2258] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:44:55.402779 13272 ts_tablet_manager.cc:1905] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:44:55.404696 13272 ts_tablet_manager.cc:1918] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:44:55.404873 13272 log.cc:1196] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/fa94ad25408043cea5173d7578521047
I20260805 22:44:55.405143 13272 ts_tablet_manager.cc:1939] T fa94ad25408043cea5173d7578521047 P bc1996c8f1f841d5a0c3709f7fdc834d: Deleting consensus metadata
I20260805 22:44:55.405651 12946 catalog_manager.cc:4841] TS bc1996c8f1f841d5a0c3709f7fdc834d (127.23.197.65:33061): tablet fa94ad25408043cea5173d7578521047 (table test-table [id=a82d7c718ec14ee7b420e53e602a7cfb]) successfully deleted
W20260805 22:44:55.458025 13257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:55.458230 13257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:55.458253 13257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:55.459347 13257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:55.459384 13257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:55.459409 13257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:44:55.460816 13257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:40115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=38457
--tserver_master_addrs=127.23.197.126:41091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:55.461004 13257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:55.461175 13257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:55.466207 13277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.466199 13280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.466210 13287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:55.466590 13257 server_base.cc:1034] running on GCE node
I20260805 22:44:55.466873 13257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:55.467167 13257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:55.468350 13257 hybrid_clock.cc:648] HybridClock initialized: now 1785969895468348 us; error 41 us; skew 500 ppm
Aug 05 22:44:55 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969895, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:55.471202 13257 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:44:55.471633 13257 webserver.cc:460] Webserver started at http://127.23.197.66:38457/ using document root <none> and password file <none>
I20260805 22:44:55.471881 13257 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:55.471977 13257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:55.473508 13257 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:44:55.474097 13295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:55.474301 13257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.474397 13257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "61c2c7417b27442686e8cbe4ad7ab785"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "3710b77c33adf0a7f5642c3896506f9e"
server_key_iv: "506d6d915d882a0edc8cab5e167a3c39"
server_key_version: "encryptionkey@0"
I20260805 22:44:55.474736 13257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:55.483469 13257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:55.485667 13257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:55.485790 13257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:55.485980 13257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:55.486320 13257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:55.486362 13257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:55.486385 13257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:55.486431 13257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:55.492532 13257 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:40115
I20260805 22:44:55.492605 13458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:40115 every 8 connection(s)
I20260805 22:44:55.493032 13257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
Aug 05 22:44:55 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969895, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:55.502794 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 13257
I20260805 22:44:55.503087 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:40859
--local_ip_for_outbound_sockets=127.23.197.67
--tserver_master_addrs=127.23.197.126:41091
--webserver_port=44369
--webserver_interface=127.23.197.67
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:44:55.503420 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.493885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:33383 (local address 127.23.197.126:41091)
0805 22:44:55.493932 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.493934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.494205 (+ 271us) server_negotiation.cc:407] Connection header received
0805 22:44:55.494761 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.494762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.494776 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.494829 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:55.495815 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.496323 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.497435 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.497649 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.497740 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.500085 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:55.500086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:55.500086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:55.500095 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:55.501901 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.502312 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.502313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.502313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.502374 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.502632 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.502633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.502633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.502787 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:55.502865 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.503171 (+ 306us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.503284 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:44:55.503556 13462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.493873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:33383)
0805 22:44:55.494068 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.494106 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:55.494632 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:55.494905 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:55.494909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:55.495494 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:55.495707 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.495712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.496425 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.496426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.497328 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.497331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.497750 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.497760 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.497887 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.498386 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:55.498395 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:55.499969 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:55.501991 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:55.501998 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:55.502000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:55.502223 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:55.502442 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:55.502442 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:55.502443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:55.502542 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:55.502882 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:55.502889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:55.503030 (+ 141us) client_negotiation.cc:769] Sending connection context
0805 22:44:55.503118 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 22:44:55.503318 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260805 22:44:55.504164 13460 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41091
I20260805 22:44:55.504292 13460 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:55.504495 13460 heartbeater.cc:511] Master 127.23.197.126:41091 requested a full tablet report, sending...
I20260805 22:44:55.504896 12977 ts_manager.cc:194] Registered new tserver with Master: 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115)
I20260805 22:44:55.505290 12977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:33383
W20260805 22:44:55.588562 13464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:55.588811 13464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:55.588855 13464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:55.589967 13464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:55.590006 13464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:55.590034 13464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:44:55.591429 13464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:40859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=44369
--tserver_master_addrs=127.23.197.126:41091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:55.591610 13464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:55.591789 13464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:55.596818 13478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.596829 13473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.596866 13475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:55.597065 13464 server_base.cc:1034] running on GCE node
I20260805 22:44:55.597317 13464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:55.597592 13464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:55.598796 13464 hybrid_clock.cc:648] HybridClock initialized: now 1785969895598780 us; error 33 us; skew 500 ppm
Aug 05 22:44:55 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969895, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:55.601950 13464 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:44:55.602348 13464 webserver.cc:460] Webserver started at http://127.23.197.67:44369/ using document root <none> and password file <none>
I20260805 22:44:55.602527 13464 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:55.602581 13464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:55.603927 13464 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.604552 13487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:55.604806 13464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.604907 13464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "a6923f848bd64c2ea3a94d2b2aaade02"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "c48ee41b83e63d165b0765d894a5ef59"
server_key_iv: "8e6609427e0305d2048b58e39f1e0212"
server_key_version: "encryptionkey@0"
I20260805 22:44:55.605255 13464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:55.630995 13464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:55.633667 13464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:55.633852 13464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:55.634115 13464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:55.634496 13464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:55.634533 13464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:55.634555 13464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:55.634603 13464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:55.640568 13464 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:40859
I20260805 22:44:55.640647 13656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:40859 every 8 connection(s)
I20260805 22:44:55.641052 13464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
Aug 05 22:44:55 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969895, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:44:55.649905 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 13464
I20260805 22:44:55.650483 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.641813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:48971 (local address 127.23.197.126:41091)
0805 22:44:55.641889 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.641892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.642156 (+ 264us) server_negotiation.cc:407] Connection header received
0805 22:44:55.642622 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.642623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.642634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.642724 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:55.643609 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.644123 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.645180 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.645445 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.645505 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.647604 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:55.647605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:55.647606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:55.647614 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:55.648922 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.649359 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.649361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.649361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.649428 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.649683 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.649684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.649685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.649828 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:55.649902 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.650273 (+ 371us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.650366 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 22:44:55.650683 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.641824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:48971)
0805 22:44:55.642024 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.642063 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:55.642541 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:55.642748 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:55.642753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:55.643340 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:55.643533 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.643537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.644199 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.644200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.645103 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.645106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.645530 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.645532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.645636 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.646079 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:55.646087 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:55.647494 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:55.649002 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:55.649015 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:55.649017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:55.649247 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:55.649497 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:55.649498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:55.649498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:55.649607 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:55.649906 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:55.649914 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:55.650082 (+ 168us) client_negotiation.cc:769] Sending connection context
0805 22:44:55.650229 (+ 147us) client_negotiation.cc:240] Negotiation successful
0805 22:44:55.650419 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260805 22:44:55.651268 13658 heartbeater.cc:347] Connected to a master server at 127.23.197.126:41091
I20260805 22:44:55.651384 13658 heartbeater.cc:464] Registering TS with master...
I20260805 22:44:55.651564 13658 heartbeater.cc:511] Master 127.23.197.126:41091 requested a full tablet report, sending...
I20260805 22:44:55.652012 12977 ts_manager.cc:194] Registered new tserver with Master: a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859)
I20260805 22:44:55.652433 12977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:48971
I20260805 22:44:55.656494 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.650341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.23.197.126:41091)
0805 22:44:55.650610 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.650612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.650619 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:44:55.650640 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.650641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.650651 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.650711 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:55.651527 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.652185 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.653101 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.653278 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.653333 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.654427 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:55.654428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:55.654428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:55.654434 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:55.655542 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.655890 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.655891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.655891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.655943 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:55.656111 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:55.656112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:55.656112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:55.656203 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:55.656324 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.656380 (+ 56us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.656435 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":71,"threads_started":1}
I20260805 22:44:55.657371 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:44:55.660260 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.657739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46004 (local address 127.23.197.126:41091)
0805 22:44:55.657879 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.657886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.657968 (+ 82us) server_negotiation.cc:407] Connection header received
0805 22:44:55.658055 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.658055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.658091 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.658176 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:44:55.658291 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.658741 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.659508 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.659701 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.659782 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.659950 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:44:55.660047 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:44:55.660143 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.660163 (+ 20us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.660179 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260805 22:44:55.660568 12977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:46004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:44:55.662245 12977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:44:55.664563 13153 tablet_service.cc:1467] Processing CreateTablet for tablet f7fc255f46e649538e6235e4534826a2 (DEFAULT_TABLE table=test-table [id=5f1be46d796e4d17bd56f0238c1c6951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:44:55.664853 13153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7fc255f46e649538e6235e4534826a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:55.666445 13110 tablet_bootstrap.cc:492] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Bootstrap starting.
I20260805 22:44:55.666759 13110 tablet_bootstrap.cc:654] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:55.667677 13110 tablet_bootstrap.cc:492] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: No bootstrap required, opened a new log
I20260805 22:44:55.667753 13110 ts_tablet_manager.cc:1397] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.667903 13110 raft_consensus.cc:348] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:55.667979 13110 raft_consensus.cc:374] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:55.668023 13110 raft_consensus.cc:729] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc1996c8f1f841d5a0c3709f7fdc834d, State: Initialized, Role: FOLLOWER
I20260805 22:44:55.668084 13110 consensus_queue.cc:260] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:55.668208 13110 ts_tablet_manager.cc:1428] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260805 22:44:55.668464 13270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.664120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:40859 (local address 127.0.0.1:36774)
0805 22:44:55.664230 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.664241 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:55.664277 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:55.664654 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:55.664655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:55.664657 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:55.664713 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.664715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.665910 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.665911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.667116 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.667120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.668167 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.668168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.668265 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.668288 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 22:44:55.668356 (+ 68us) client_negotiation.cc:240] Negotiation successful
0805 22:44:55.668414 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260805 22:44:55.668797 13668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.664185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36774 (local address 127.23.197.67:40859)
0805 22:44:55.664478 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.664483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.664492 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:44:55.664527 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.664529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.664545 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.664639 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:55.664827 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.665801 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.667223 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.668041 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.668159 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.668228 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.668638 (+ 410us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.668686 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260805 22:44:55.669577 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.664170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40115 (local address 127.0.0.1:51546)
0805 22:44:55.664381 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.664391 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:55.664442 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:55.664752 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:55.664753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:55.664755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:55.664825 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.664827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.666109 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.666109 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.667141 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.667143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.669279 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.669281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.669351 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.669355 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:44:55.669442 (+ 87us) client_negotiation.cc:240] Negotiation successful
0805 22:44:55.669485 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":127,"threads_started":1}
I20260805 22:44:55.669579 13670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.664286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51546 (local address 127.23.197.66:40115)
0805 22:44:55.664551 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.664555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.664563 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:44:55.664593 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.664594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.664628 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.664737 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:55.664927 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.666002 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.667491 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.669128 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.669263 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.669322 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.669461 (+ 139us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.669498 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":85,"threads_started":1}
I20260805 22:44:55.669544 13554 tablet_service.cc:1467] Processing CreateTablet for tablet f7fc255f46e649538e6235e4534826a2 (DEFAULT_TABLE table=test-table [id=5f1be46d796e4d17bd56f0238c1c6951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:44:55.669942 13554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7fc255f46e649538e6235e4534826a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:55.670306 13358 tablet_service.cc:1467] Processing CreateTablet for tablet f7fc255f46e649538e6235e4534826a2 (DEFAULT_TABLE table=test-table [id=5f1be46d796e4d17bd56f0238c1c6951]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:44:55.670641 13358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7fc255f46e649538e6235e4534826a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:44:55.673722 13672 tablet_bootstrap.cc:492] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: Bootstrap starting.
I20260805 22:44:55.674319 13673 tablet_bootstrap.cc:492] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: Bootstrap starting.
I20260805 22:44:55.674438 13672 tablet_bootstrap.cc:654] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:55.675053 13673 tablet_bootstrap.cc:654] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: Neither blocks nor log segments found. Creating new log.
I20260805 22:44:55.675053 13672 log.cc:824] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:55.675662 13673 log.cc:824] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:55.675930 13672 tablet_bootstrap.cc:492] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: No bootstrap required, opened a new log
I20260805 22:44:55.676115 13672 ts_tablet_manager.cc:1397] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:44:55.676483 13673 tablet_bootstrap.cc:492] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: No bootstrap required, opened a new log
I20260805 22:44:55.676559 13673 ts_tablet_manager.cc:1397] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:44:55.678262 13672 raft_consensus.cc:348] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:55.678463 13672 raft_consensus.cc:374] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:55.678504 13672 raft_consensus.cc:729] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6923f848bd64c2ea3a94d2b2aaade02, State: Initialized, Role: FOLLOWER
I20260805 22:44:55.678606 13673 raft_consensus.cc:348] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:55.678668 13672 consensus_queue.cc:260] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:55.678758 13673 raft_consensus.cc:374] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:44:55.678798 13673 raft_consensus.cc:729] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c2c7417b27442686e8cbe4ad7ab785, State: Initialized, Role: FOLLOWER
I20260805 22:44:55.678923 13673 consensus_queue.cc:260] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:55.678969 13672 ts_tablet_manager.cc:1428] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:44:55.679100 13658 heartbeater.cc:503] Master 127.23.197.126:41091 was elected leader, sending a full tablet report...
I20260805 22:44:55.679208 13673 ts_tablet_manager.cc:1428] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:44:55.679229 13460 heartbeater.cc:503] Master 127.23.197.126:41091 was elected leader, sending a full tablet report...
I20260805 22:44:55.694067 13676 raft_consensus.cc:482] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:44:55.694221 13676 raft_consensus.cc:504] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:55.694675 13676 leader_election.cc:288] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115), bc1996c8f1f841d5a0c3709f7fdc834d (127.23.197.65:33061)
I20260805 22:44:55.697741 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.694829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40115 (local address 127.23.197.67:35687)
0805 22:44:55.694885 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.694897 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:55.694948 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:55.695172 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:55.695174 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:55.695176 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:55.695273 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.695275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.696003 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.696004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.696882 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.696884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.697468 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.697469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.697562 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.697567 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:44:55.697623 (+ 56us) client_negotiation.cc:240] Negotiation successful
0805 22:44:55.697666 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260805 22:44:55.697752 13670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.694942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:35687 (local address 127.23.197.66:40115)
0805 22:44:55.695003 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.695005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.695015 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:44:55.695045 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.695046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.695056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.695121 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:55.695358 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.695918 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.696965 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.697387 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.697444 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.697451 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.697643 (+ 192us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.697681 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 22:44:55.697932 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.694914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33061 (local address 127.23.197.67:51683)
0805 22:44:55.695094 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.695105 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:55.695168 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:55.695357 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:55.695357 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:55.695359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:44:55.695431 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.695432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.696132 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.696133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.696983 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:55.696985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.697653 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.697654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:55.697729 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.697733 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:44:55.697794 (+ 61us) client_negotiation.cc:240] Negotiation successful
0805 22:44:55.697837 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":115,"threads_started":1}
I20260805 22:44:55.697942 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.695062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:51683 (local address 127.23.197.65:33061)
0805 22:44:55.695108 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.695111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.695172 (+ 61us) server_negotiation.cc:407] Connection header received
0805 22:44:55.695230 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.695231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.695262 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.695337 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:44:55.695506 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.696049 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.697070 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.697541 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.697622 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.697723 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.697806 (+ 83us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.697864 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:44:55.698153 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7fc255f46e649538e6235e4534826a2" candidate_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" is_pre_election: true
I20260805 22:44:55.698122 13391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7fc255f46e649538e6235e4534826a2" candidate_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61c2c7417b27442686e8cbe4ad7ab785" is_pre_election: true
I20260805 22:44:55.698302 13391 raft_consensus.cc:2454] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6923f848bd64c2ea3a94d2b2aaade02 in term 0.
I20260805 22:44:55.698319 13199 raft_consensus.cc:2454] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6923f848bd64c2ea3a94d2b2aaade02 in term 0.
I20260805 22:44:55.698586 13489 leader_election.cc:302] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6923f848bd64c2ea3a94d2b2aaade02, bc1996c8f1f841d5a0c3709f7fdc834d; no voters:
I20260805 22:44:55.698776 13676 raft_consensus.cc:2781] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:44:55.698863 13676 raft_consensus.cc:482] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:44:55.698900 13676 raft_consensus.cc:3037] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:55.699626 13676 raft_consensus.cc:504] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:55.699808 13676 leader_election.cc:288] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [CANDIDATE]: Term 1 election: Requested vote from peers 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115), bc1996c8f1f841d5a0c3709f7fdc834d (127.23.197.65:33061)
I20260805 22:44:55.699962 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7fc255f46e649538e6235e4534826a2" candidate_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d"
I20260805 22:44:55.699950 13391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7fc255f46e649538e6235e4534826a2" candidate_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61c2c7417b27442686e8cbe4ad7ab785"
I20260805 22:44:55.700063 13199 raft_consensus.cc:3037] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:55.700058 13391 raft_consensus.cc:3037] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:44:55.700830 13199 raft_consensus.cc:2454] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6923f848bd64c2ea3a94d2b2aaade02 in term 1.
I20260805 22:44:55.700896 13391 raft_consensus.cc:2454] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6923f848bd64c2ea3a94d2b2aaade02 in term 1.
I20260805 22:44:55.701038 13489 leader_election.cc:302] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6923f848bd64c2ea3a94d2b2aaade02, bc1996c8f1f841d5a0c3709f7fdc834d; no voters:
I20260805 22:44:55.701148 13676 raft_consensus.cc:2781] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:44:55.701344 13676 raft_consensus.cc:686] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 1 LEADER]: Becoming Leader. State: Replica: a6923f848bd64c2ea3a94d2b2aaade02, State: Running, Role: LEADER
I20260805 22:44:55.701445 13676 consensus_queue.cc:237] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:55.702294 12973 catalog_manager.cc:5495] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67). New cstate: current_term: 1 leader_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } health_report { overall_health: UNKNOWN } } }
I20260805 22:44:55.713958 13668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.711759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.23.197.67:40859)
0805 22:44:55.711826 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:55.711829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:55.711874 (+ 45us) server_negotiation.cc:407] Connection header received
0805 22:44:55.711948 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:55.711949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:55.711965 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:55.712015 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:44:55.712114 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.712553 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.713285 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:55.713465 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:55.713521 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:55.713631 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:44:55.713719 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:44:55.713779 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:55.713821 (+ 42us) server_negotiation.cc:299] Negotiation successful
0805 22:44:55.713835 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 22:44:55.715803 13391 raft_consensus.cc:1261] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [term 1 FOLLOWER]: Refusing update from remote peer a6923f848bd64c2ea3a94d2b2aaade02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:44:55.715801 13199 raft_consensus.cc:1261] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Refusing update from remote peer a6923f848bd64c2ea3a94d2b2aaade02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:44:55.716182 13681 consensus_queue.cc:1035] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:44:55.716325 13676 consensus_queue.cc:1035] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:44:55.719291 13688 mvcc.cc:205] Tried to move back new op lower bound from 7315332692849078272 to 7315332692793839616. Current Snapshot: MvccSnapshot[applied={T|T < 7315332692849078272}]
I20260805 22:44:55.719504 13685 mvcc.cc:205] Tried to move back new op lower bound from 7315332692849078272 to 7315332692793839616. Current Snapshot: MvccSnapshot[applied={T|T < 7315332692849078272}]
I20260805 22:44:55.721011 12977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:46004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:44:55.721124 12977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:46004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:44:55.721915 12977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Sending DeleteTablet for 3 replicas of tablet f7fc255f46e649538e6235e4534826a2
I20260805 22:44:55.722267 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 13043
I20260805 22:44:55.722293 13358 tablet_service.cc:1514] Processing DeleteTablet for tablet f7fc255f46e649538e6235e4534826a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:44:55 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:51546
I20260805 22:44:55.736702 13700 tablet_replica.cc:330] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: stopping tablet replica
I20260805 22:44:55.736965 13700 raft_consensus.cc:2229] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:44:55.737115 13700 raft_consensus.cc:2258] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:44:55.737798 13700 ts_tablet_manager.cc:1905] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:44:55.739933 13700 ts_tablet_manager.cc:1918] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:44:55.740042 13700 log.cc:1196] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/f7fc255f46e649538e6235e4534826a2
I20260805 22:44:55.740336 13700 ts_tablet_manager.cc:1939] T f7fc255f46e649538e6235e4534826a2 P 61c2c7417b27442686e8cbe4ad7ab785: Deleting consensus metadata
I20260805 22:44:55.742033 13554 tablet_service.cc:1514] Processing DeleteTablet for tablet f7fc255f46e649538e6235e4534826a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:44:55 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:36774
I20260805 22:44:55.742383 12947 catalog_manager.cc:4841] TS 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115): tablet f7fc255f46e649538e6235e4534826a2 (table test-table [id=5f1be46d796e4d17bd56f0238c1c6951]) successfully deleted
I20260805 22:44:55.743149 13701 tablet_replica.cc:330] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: stopping tablet replica
I20260805 22:44:55.743300 13701 raft_consensus.cc:2229] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:44:55.743430 13701 raft_consensus.cc:2258] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:44:55.743993 13701 ts_tablet_manager.cc:1905] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:44:55.745797 13701 ts_tablet_manager.cc:1918] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:44:55.745889 13701 log.cc:1196] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/f7fc255f46e649538e6235e4534826a2
I20260805 22:44:55.746284 13701 ts_tablet_manager.cc:1939] T f7fc255f46e649538e6235e4534826a2 P a6923f848bd64c2ea3a94d2b2aaade02: Deleting consensus metadata
I20260805 22:44:55.746865 12946 catalog_manager.cc:4841] TS a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859): tablet f7fc255f46e649538e6235e4534826a2 (table test-table [id=5f1be46d796e4d17bd56f0238c1c6951]) successfully deleted
W20260805 22:44:55.747154 12946 connection.cc:535] client connection to 127.23.197.65:33061 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260805 22:44:55.747273 12946 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 22:44:55.747665 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 13257
I20260805 22:44:55.747824 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:55.747660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33061 (local address 127.0.0.1:38080)
0805 22:44:55.747744 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:55.747784 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260805 22:44:55.752013 12946 catalog_manager.cc:4584] TS bc1996c8f1f841d5a0c3709f7fdc834d (127.23.197.65:33061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7fc255f46e649538e6235e4534826a2: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33061: connect: Connection refused (error 111)
I20260805 22:44:55.775732 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 13464
I20260805 22:44:55.789276 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 12922
I20260805 22:44:55.802289 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41091
--webserver_interface=127.23.197.126
--webserver_port=41437
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:41091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:55.891856 13703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:55.892153 13703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:55.892202 13703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:55.893497 13703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:44:55.893581 13703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:55.893651 13703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:55.893682 13703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:44:55.893708 13703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:44:55.895217 13703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:41091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=41437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:55.895503 13703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:55.895751 13703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:55.900955 13709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.900965 13710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:55.900985 13713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:55.901338 13703 server_base.cc:1034] running on GCE node
I20260805 22:44:55.901594 13703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:55.901916 13703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:55.903095 13703 hybrid_clock.cc:648] HybridClock initialized: now 1785969895903074 us; error 39 us; skew 500 ppm
Aug 05 22:44:55 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969895, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:55.905859 13703 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:44:55.906373 13703 webserver.cc:460] Webserver started at http://127.23.197.126:41437/ using document root <none> and password file <none>
I20260805 22:44:55.906607 13703 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:55.906688 13703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:55.908224 13703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:44:55.909288 13724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:55.909525 13703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:55.909651 13703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "083b54d10b394efda4e36d8b7181292e"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "fa1f8540e7a1c94232b91fa1cd01d0a1"
server_key_iv: "e10c2ed0353f04b3b5dcc60e6fa69c55"
server_key_version: "encryptionkey@0"
I20260805 22:44:55.910090 13703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:55.964133 13703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:55.966928 13703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:55.967119 13703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:55.972139 13703 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:41091
I20260805 22:44:55.972199 13809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:41091 every 8 connection(s)
I20260805 22:44:55.972658 13703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:44:55.974490 13811 sys_catalog.cc:263] Verifying existing consensus state
I20260805 22:44:55.975260 13811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Bootstrap starting.
I20260805 22:44:55.978735 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 13703
I20260805 22:44:55.979149 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:33061
--local_ip_for_outbound_sockets=127.23.197.65
--tserver_master_addrs=127.23.197.126:41091
--webserver_port=38199
--webserver_interface=127.23.197.65
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:44:55.985873 13811 log.cc:824] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:55.988416 13811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:44:55.988832 13811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e: Bootstrap complete.
I20260805 22:44:55.991416 13811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:55.991696 13811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083b54d10b394efda4e36d8b7181292e, State: Initialized, Role: FOLLOWER
I20260805 22:44:55.991930 13811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:55.992039 13811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:44:55.992101 13811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:44:55.992156 13811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 2 FOLLOWER]: Advancing to term 3
I20260805 22:44:55.993347 13811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:55.993481 13811 leader_election.cc:302] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 083b54d10b394efda4e36d8b7181292e; no voters:
I20260805 22:44:55.993705 13811 leader_election.cc:288] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260805 22:44:55.993777 13816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 3 FOLLOWER]: Leader election won for term 3
I20260805 22:44:55.993945 13816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [term 3 LEADER]: Becoming Leader. State: Replica: 083b54d10b394efda4e36d8b7181292e, State: Running, Role: LEADER
I20260805 22:44:55.994060 13811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:44:55.994148 13816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } }
I20260805 22:44:55.994477 13816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 083b54d10b394efda4e36d8b7181292e. Latest consensus state: current_term: 3 leader_uuid: "083b54d10b394efda4e36d8b7181292e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } } }
I20260805 22:44:55.994479 13818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "083b54d10b394efda4e36d8b7181292e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083b54d10b394efda4e36d8b7181292e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41091 } } }
I20260805 22:44:55.994647 13816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:55.994663 13818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083b54d10b394efda4e36d8b7181292e [sys.catalog]: This master's current role is: LEADER
I20260805 22:44:55.994861 13827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:44:55.995791 13827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:44:55.995913 13827 catalog_manager.cc:1259] Loaded cluster ID: 026cfd4ad6da4235a388815b879ea555
I20260805 22:44:55.995951 13827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260805 22:44:55.996309 13827 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa3ceb1c9ad google::LogMessage::Fail() at ??:0
@ 0x7fa3ceb1f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa3ceb1c4de google::LogMessage::Flush() at ??:0
@ 0x7fa3ceb1da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa3d6a3d0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa3d6a468b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa3d09ea0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa3d09df954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa3d18436db start_thread at ??:0
@ 0x7fa3cd61a61f clone at ??:0
W20260805 22:44:56.086998 13814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:56.087193 13814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:56.087215 13814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:56.088372 13814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:56.088419 13814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:56.088446 13814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:44:56.089986 13814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:33061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=38199
--tserver_master_addrs=127.23.197.126:41091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:56.090242 13814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:56.090490 13814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:56.096835 13867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:56.096829 13868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:56.096845 13873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:56.097031 13814 server_base.cc:1034] running on GCE node
I20260805 22:44:56.097343 13814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:56.097623 13814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:56.098831 13814 hybrid_clock.cc:648] HybridClock initialized: now 1785969896098812 us; error 33 us; skew 500 ppm
Aug 05 22:44:56 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969896, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:56.101475 13814 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:44:56.101969 13814 webserver.cc:460] Webserver started at http://127.23.197.65:38199/ using document root <none> and password file <none>
I20260805 22:44:56.102219 13814 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:56.102293 13814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:56.103755 13814 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:56.104537 13882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:56.104753 13814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:44:56.104893 13814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "bc1996c8f1f841d5a0c3709f7fdc834d"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "149cc5de35c2f745c12f06edb5fa7e1c"
server_key_iv: "1bfb389b57a5692345cd73dc3ec34b63"
server_key_version: "encryptionkey@0"
I20260805 22:44:56.105301 13814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:56.123584 13814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:56.126088 13814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:56.126247 13814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:56.126520 13814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:56.127007 13892 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 22:44:56.129019 13814 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 22:44:56.129086 13814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260805 22:44:56.129136 13814 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 22:44:56.129803 13814 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 22:44:56.129863 13814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260805 22:44:56.129946 13892 tablet_bootstrap.cc:492] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Bootstrap starting.
I20260805 22:44:56.137739 13814 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:33061
I20260805 22:44:56.138137 13814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:44:56.138527 14048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:33061 every 8 connection(s)
I20260805 22:44:56.141295 13892 log.cc:824] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Log is configured to *not* fsync() on all Append() calls
I20260805 22:44:56.144222 13892 tablet_bootstrap.cc:492] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:44:56.144562 13892 tablet_bootstrap.cc:492] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Bootstrap complete.
I20260805 22:44:56.145859 13892 ts_tablet_manager.cc:1397] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.010s
I20260805 22:44:56.145942 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 13814
I20260805 22:44:56.146214 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:40115
--local_ip_for_outbound_sockets=127.23.197.66
--tserver_master_addrs=127.23.197.126:41091
--webserver_port=38457
--webserver_interface=127.23.197.66
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:44:56.147270 13892 raft_consensus.cc:348] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:56.147603 13892 raft_consensus.cc:729] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc1996c8f1f841d5a0c3709f7fdc834d, State: Initialized, Role: FOLLOWER
I20260805 22:44:56.147796 13892 consensus_queue.cc:260] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:56.148128 13892 ts_tablet_manager.cc:1428] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260805 22:44:56.202585 14052 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:44:56.138861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:33519)
0805 22:44:56.139047 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:56.139094 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:56.139702 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:56.202363 (+ 62661us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
W20260805 22:44:56.202894 14049 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:41091 (0 consecutive failures): Network error: Failed to ping master at 127.23.197.126:41091: Client connection negotiation failed: client connection to 127.23.197.126:41091: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 22:44:56.203258 14052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:56.203139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:53297)
0805 22:44:56.203194 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:56.203239 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260805 22:44:56.203774 14052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:56.203627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:40225)
0805 22:44:56.203707 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:56.203744 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260805 22:44:56.203999 14049 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 22:44:56.236423 14057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:56.236657 14057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:56.236698 14057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:56.237882 14057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:56.237953 14057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:56.237995 14057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:44:56.239499 14057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:40115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=38457
--tserver_master_addrs=127.23.197.126:41091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:56.239776 14057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:56.240042 14057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:56.245081 14065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:56.245093 14068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:56.245093 14066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:56.245581 14057 server_base.cc:1034] running on GCE node
I20260805 22:44:56.245793 14057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:56.246012 14057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:56.247197 14057 hybrid_clock.cc:648] HybridClock initialized: now 1785969896247193 us; error 40 us; skew 500 ppm
Aug 05 22:44:56 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969896, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:56.249859 14057 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:44:56.250353 14057 webserver.cc:460] Webserver started at http://127.23.197.66:38457/ using document root <none> and password file <none>
I20260805 22:44:56.250608 14057 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:56.250682 14057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:56.252422 14057 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:44:56.253342 14081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:56.253563 14057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:44:56.253691 14057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "61c2c7417b27442686e8cbe4ad7ab785"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "3710b77c33adf0a7f5642c3896506f9e"
server_key_iv: "506d6d915d882a0edc8cab5e167a3c39"
server_key_version: "encryptionkey@0"
I20260805 22:44:56.254123 14057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:56.271662 14057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:56.274915 14057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:56.275148 14057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:56.275465 14057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:56.275856 14057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:56.275903 14057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:56.275944 14057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:56.275964 14057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:56.282049 14057 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:40115
I20260805 22:44:56.282135 14244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:40115 every 8 connection(s)
I20260805 22:44:56.282493 14057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:44:56.283694 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:56.283241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:48415)
0805 22:44:56.283410 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:56.283495 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
W20260805 22:44:56.284233 14245 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:41091 (0 consecutive failures): Network error: Failed to ping master at 127.23.197.126:41091: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
I20260805 22:44:56.284590 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:56.284458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:57841)
0805 22:44:56.284525 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:56.284568 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260805 22:44:56.285084 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:56.284992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:57661)
0805 22:44:56.285040 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:56.285066 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260805 22:44:56.285254 14245 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 22:44:56.292709 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 14057
I20260805 22:44:56.293071 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:40859
--local_ip_for_outbound_sockets=127.23.197.67
--tserver_master_addrs=127.23.197.126:41091
--webserver_port=44369
--webserver_interface=127.23.197.67
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:44:56.384804 14249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:44:56.385030 14249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:44:56.385058 14249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:44:56.386372 14249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:44:56.386425 14249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:44:56.386453 14249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:44:56.388120 14249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:40859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=44369
--tserver_master_addrs=127.23.197.126:41091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:44:56.388408 14249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:44:56.388686 14249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:44:56.394245 14261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:44:56.394289 14266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:56.394563 14249 server_base.cc:1034] running on GCE node
W20260805 22:44:56.394268 14260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:44:56.394836 14249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:44:56.395149 14249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:44:56.396333 14249 hybrid_clock.cc:648] HybridClock initialized: now 1785969896396295 us; error 51 us; skew 500 ppm
Aug 05 22:44:56 dist-test-slave-g170 krb5kdc[12079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969896, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:44:56.399369 14249 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:44:56.399850 14249 webserver.cc:460] Webserver started at http://127.23.197.67:44369/ using document root <none> and password file <none>
I20260805 22:44:56.400103 14249 fs_manager.cc:359] Metadata directory not provided
I20260805 22:44:56.400177 14249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:44:56.401615 14249 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:44:56.402336 14275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:56.402534 14249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:44:56.402637 14249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "a6923f848bd64c2ea3a94d2b2aaade02"
format_stamp: "Formatted at 2026-08-05 22:44:54 on dist-test-slave-g170"
server_key: "c48ee41b83e63d165b0765d894a5ef59"
server_key_iv: "8e6609427e0305d2048b58e39f1e0212"
server_key_version: "encryptionkey@0"
I20260805 22:44:56.402983 14249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:44:56.415634 14249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:44:56.418095 14249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:44:56.418267 14249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:44:56.418522 14249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:44:56.418838 14249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:44:56.418895 14249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:56.418936 14249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:44:56.418964 14249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:44:56.425150 14249 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:40859
I20260805 22:44:56.425216 14469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:40859 every 8 connection(s)
I20260805 22:44:56.425598 14249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:44:56.426720 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:56.426316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:44591)
0805 22:44:56.426481 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:56.426564 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
W20260805 22:44:56.427182 14471 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:41091 (0 consecutive failures): Network error: Failed to ping master at 127.23.197.126:41091: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
I20260805 22:44:56.427515 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:56.427375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:59933)
0805 22:44:56.427449 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:56.427483 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260805 22:44:56.427892 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:56.427786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:37615)
0805 22:44:56.427842 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:56.427870 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260805 22:44:56.428041 14471 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 22:44:56.429319 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 14249
I20260805 22:44:57.204689 14477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:57.204377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:35151)
0805 22:44:57.204542 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:57.204580 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
I20260805 22:44:57.285980 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:57.285686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:48171)
0805 22:44:57.285853 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:57.285891 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":67,"threads_started":1}
I20260805 22:44:57.317396 14480 raft_consensus.cc:482] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:44:57.317526 14480 raft_consensus.cc:504] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:57.317869 14480 leader_election.cc:288] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859), 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115)
Aug 05 22:44:57 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969896, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.67@KRBTEST.COM
Aug 05 22:44:57 dist-test-slave-g170 krb5kdc[12079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969896, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.66@KRBTEST.COM
I20260805 22:44:57.328987 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:57.318110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:40859 (local address 127.23.197.65:38331)
0805 22:44:57.318222 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:57.318233 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:57.318291 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:57.319203 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:57.319204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:57.319630 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:57.319817 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:57.319822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:57.321218 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:57.321219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:57.321954 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:57.321957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:57.322649 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:57.322650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:57.322760 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:57.323454 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:57.323459 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:57.325088 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:57.327266 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:57.327305 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:57.327307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:57.327617 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:57.327919 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:57.327919 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:57.327920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:57.328049 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:57.328435 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:57.328443 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:57.328593 (+ 150us) client_negotiation.cc:769] Sending connection context
0805 22:44:57.328677 (+ 84us) client_negotiation.cc:240] Negotiation successful
0805 22:44:57.328866 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":64,"thread_start_us":53,"threads_started":1}
I20260805 22:44:57.329267 14477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:57.318087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:40115 (local address 127.23.197.65:49103)
0805 22:44:57.318154 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:57.318170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 22:44:57.318249 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:44:57.319191 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:44:57.319193 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:44:57.319628 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:44:57.319816 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:57.319822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:57.321148 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:57.321150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:57.321952 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:44:57.321954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:44:57.322648 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:44:57.322649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:44:57.322759 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:57.323447 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:44:57.323455 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:44:57.325431 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:44:57.327884 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:57.327885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:57.327885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:57.328109 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:57.328351 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:44:57.328352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:44:57.328353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:44:57.328442 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:44:57.328941 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:44:57.328942 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:44:57.328984 (+ 42us) client_negotiation.cc:769] Sending connection context
0805 22:44:57.329062 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 22:44:57.329175 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":3}
I20260805 22:44:57.329070 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:57.318320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:38331 (local address 127.23.197.67:40859)
0805 22:44:57.318497 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:57.318509 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:57.318536 (+ 27us) server_negotiation.cc:407] Connection header received
0805 22:44:57.318971 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:57.318974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:57.319024 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:57.319179 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:57.319945 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:57.321123 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:57.322069 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:57.322551 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:57.322683 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:57.325212 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:57.325217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:57.325218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:57.325257 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:57.327159 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:57.327730 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:57.327733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:57.327734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:57.327865 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:57.328122 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:57.328122 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:57.328122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:57.328367 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:57.328430 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:57.328785 (+ 355us) server_negotiation.cc:299] Negotiation successful
0805 22:44:57.328941 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260805 22:44:57.329403 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:57.318240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:49103 (local address 127.23.197.66:40115)
0805 22:44:57.318394 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 22:44:57.318409 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 22:44:57.318439 (+ 30us) server_negotiation.cc:407] Connection header received
0805 22:44:57.318949 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:44:57.318952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:44:57.318995 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:44:57.319109 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:44:57.319954 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:57.321050 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:57.322040 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:44:57.322551 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:44:57.322675 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:44:57.325539 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:44:57.325550 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:44:57.325550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:44:57.325561 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:44:57.327789 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:57.328175 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:57.328179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:57.328179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:57.328268 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:44:57.328498 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:44:57.328499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:44:57.328499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:44:57.328862 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:44:57.328952 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 22:44:57.329141 (+ 189us) server_negotiation.cc:299] Negotiation successful
0805 22:44:57.329287 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
W20260805 22:44:57.330181 13883 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
W20260805 22:44:57.330291 13884 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
I20260805 22:44:57.330320 13883 leader_election.cc:302] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc1996c8f1f841d5a0c3709f7fdc834d; no voters: 61c2c7417b27442686e8cbe4ad7ab785, a6923f848bd64c2ea3a94d2b2aaade02
I20260805 22:44:57.330441 14480 raft_consensus.cc:2726] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:44:57.428647 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:57.428380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:51579)
0805 22:44:57.428561 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:57.428608 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":108,"threads_started":1}
I20260805 22:44:58.205540 14490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:58.205258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:45561)
0805 22:44:58.205420 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:58.205472 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260805 22:44:58.286767 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:58.286491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:46261)
0805 22:44:58.286675 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:58.286721 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260805 22:44:58.429322 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:58.429099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:40683)
0805 22:44:58.429237 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:58.429281 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":58,"threads_started":1}
I20260805 22:44:59.206305 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:59.206028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:60153)
0805 22:44:59.206211 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:59.206256 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
I20260805 22:44:59.287573 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:59.287335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:46205)
0805 22:44:59.287490 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:59.287531 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260805 22:44:59.430171 14502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:44:59.429920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:34089)
0805 22:44:59.430080 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0805 22:44:59.430124 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260805 22:44:59.507634 14503 raft_consensus.cc:482] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:44:59.507740 14503 raft_consensus.cc:504] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:44:59.507943 14503 leader_election.cc:288] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859), 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115)
W20260805 22:44:59.508435 13883 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
W20260805 22:44:59.508518 13884 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
I20260805 22:44:59.508618 13884 leader_election.cc:302] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc1996c8f1f841d5a0c3709f7fdc834d; no voters: 61c2c7417b27442686e8cbe4ad7ab785, a6923f848bd64c2ea3a94d2b2aaade02
I20260805 22:44:59.508827 14503 raft_consensus.cc:2726] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:45:00.207193 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:00.206937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:55259)
0805 22:45:00.207087 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:00.207142 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260805 22:45:00.288375 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:00.288129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:33293)
0805 22:45:00.288284 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:00.288328 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260805 22:45:00.431063 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:00.430807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:34931)
0805 22:45:00.430966 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:00.431013 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":76,"threads_started":1}
I20260805 22:45:01.208065 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:01.207801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:58573)
0805 22:45:01.207973 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:01.208014 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260805 22:45:01.289288 14508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:01.289032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:51993)
0805 22:45:01.289197 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:01.289239 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":68,"threads_started":1}
I20260805 22:45:01.431839 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:01.431574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:35091)
0805 22:45:01.431749 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:01.431792 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260805 22:45:01.569052 14510 raft_consensus.cc:482] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:45:01.569152 14510 raft_consensus.cc:504] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:45:01.569334 14510 leader_election.cc:288] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859), 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115)
W20260805 22:45:01.569818 13883 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
W20260805 22:45:01.569901 13884 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
I20260805 22:45:01.569936 13884 leader_election.cc:302] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc1996c8f1f841d5a0c3709f7fdc834d; no voters: 61c2c7417b27442686e8cbe4ad7ab785, a6923f848bd64c2ea3a94d2b2aaade02
I20260805 22:45:01.570024 14510 raft_consensus.cc:2726] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:45:02.208940 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:02.208580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:35389)
0805 22:45:02.208841 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:02.208886 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":65,"threads_started":1}
I20260805 22:45:02.290182 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:02.289907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:33653)
0805 22:45:02.290067 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:02.290130 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":62,"threads_started":1}
I20260805 22:45:02.432657 14513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:02.432430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:42575)
0805 22:45:02.432565 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:02.432607 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260805 22:45:03.209707 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:03.209456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:40391)
0805 22:45:03.209604 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:03.209658 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":62,"threads_started":1}
I20260805 22:45:03.290999 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:03.290750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:48301)
0805 22:45:03.290909 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:03.290953 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260805 22:45:03.433467 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:03.433226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:47969)
0805 22:45:03.433372 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:03.433418 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260805 22:45:04.210516 14519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:04.210285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:35841)
0805 22:45:04.210433 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:04.210467 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260805 22:45:04.291926 14520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:04.291642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:37809)
0805 22:45:04.291802 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:04.291879 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260805 22:45:04.434270 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:04.434028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:55725)
0805 22:45:04.434182 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:04.434220 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260805 22:45:05.098649 14522 raft_consensus.cc:482] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:45:05.098785 14522 raft_consensus.cc:504] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:45:05.099016 14522 leader_election.cc:288] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859), 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115)
W20260805 22:45:05.099435 13883 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
W20260805 22:45:05.099547 13884 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
I20260805 22:45:05.099633 13884 leader_election.cc:302] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc1996c8f1f841d5a0c3709f7fdc834d; no voters: 61c2c7417b27442686e8cbe4ad7ab785, a6923f848bd64c2ea3a94d2b2aaade02
I20260805 22:45:05.099737 14522 raft_consensus.cc:2726] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:45:05.211249 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:05.211040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:50111)
0805 22:45:05.211169 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:05.211200 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260805 22:45:05.292690 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:05.292436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:43531)
0805 22:45:05.292593 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:05.292629 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260805 22:45:05.434988 14525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:05.434777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:38521)
0805 22:45:05.434901 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:05.434941 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260805 22:45:06.211967 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:06.211737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:60359)
0805 22:45:06.211879 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:06.211918 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260805 22:45:06.293677 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:06.293352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:34353)
0805 22:45:06.293553 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:06.293612 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":66,"threads_started":1}
I20260805 22:45:06.435760 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:06.435525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:57363)
0805 22:45:06.435657 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:06.435710 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260805 22:45:07.212842 14532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:07.212498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:51141)
0805 22:45:07.212672 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:07.212731 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260805 22:45:07.294487 14533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:07.294240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:37523)
0805 22:45:07.294391 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:07.294436 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260805 22:45:07.436546 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:07.436272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:34517)
0805 22:45:07.436457 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:07.436497 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260805 22:45:08.213600 14536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:08.213367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:46799)
0805 22:45:08.213504 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:08.213548 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260805 22:45:08.295252 14538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:08.295028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:39777)
0805 22:45:08.295160 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:08.295195 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260805 22:45:08.437372 14539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:08.437135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:60625)
0805 22:45:08.437284 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:08.437326 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260805 22:45:08.859464 14542 raft_consensus.cc:482] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:45:08.859565 14542 raft_consensus.cc:504] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6923f848bd64c2ea3a94d2b2aaade02" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 40859 } } peers { permanent_uuid: "61c2c7417b27442686e8cbe4ad7ab785" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 40115 } } peers { permanent_uuid: "bc1996c8f1f841d5a0c3709f7fdc834d" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33061 } }
I20260805 22:45:08.859759 14542 leader_election.cc:288] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859), 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115)
W20260805 22:45:08.860266 13883 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6923f848bd64c2ea3a94d2b2aaade02 (127.23.197.67:40859): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
W20260805 22:45:08.860373 13884 leader_election.cc:341] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61c2c7417b27442686e8cbe4ad7ab785 (127.23.197.66:40115): Not found: Tablet not found: f7fc255f46e649538e6235e4534826a2
I20260805 22:45:08.860392 13884 leader_election.cc:302] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc1996c8f1f841d5a0c3709f7fdc834d; no voters: 61c2c7417b27442686e8cbe4ad7ab785, a6923f848bd64c2ea3a94d2b2aaade02
I20260805 22:45:08.860522 14542 raft_consensus.cc:2726] T f7fc255f46e649538e6235e4534826a2 P bc1996c8f1f841d5a0c3709f7fdc834d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:45:09.214411 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:09.214177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:40059)
0805 22:45:09.214328 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:09.214364 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260805 22:45:09.296093 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:09.295774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:43469)
0805 22:45:09.296007 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:09.296044 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
I20260805 22:45:09.438187 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:09.437920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:47469)
0805 22:45:09.438079 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:09.438134 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260805 22:45:10.215235 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:10.214945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:57239)
0805 22:45:10.215124 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:10.215171 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":81,"threads_started":1}
I20260805 22:45:10.297013 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:10.296642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:40281)
0805 22:45:10.296863 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:10.296935 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":148,"threads_started":1}
I20260805 22:45:10.439064 14552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:10.438750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:57053)
0805 22:45:10.438902 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:10.438949 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
I20260805 22:45:11.216173 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:11.215932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.65:42013)
0805 22:45:11.216076 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:11.216117 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260805 22:45:11.297973 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:11.297694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.66:58651)
0805 22:45:11.297869 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:11.297921 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":75,"threads_started":1}
I20260805 22:45:11.429888 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 13814
I20260805 22:45:11.440055 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:11.439797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41091 (local address 127.23.197.67:55105)
0805 22:45:11.439966 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:11.440006 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:41091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260805 22:45:11.442521 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 14057
I20260805 22:45:11.453719 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 14249
2026-08-05T22:45:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18750 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:45:11 dist-test-slave-g170 krb5kdc[14561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:45:11 dist-test-slave-g170 krb5kdc[14561](info): set up 2 sockets
Aug 05 22:45:11 dist-test-slave-g170 krb5kdc[14561](info): commencing operation
krb5kdc: starting...
W20260805 22:45:12.715291 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:45:12 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:45:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:45:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:45:12.851889 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38715
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:38715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:45:12.936822 14578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:12.937063 14578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:12.937101 14578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:12.938244 14578 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260805 22:45:12.938308 14578 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260805 22:45:12.938346 14578 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260805 22:45:12.938366 14578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:45:12.938397 14578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:12.938423 14578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:12.938450 14578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:45:12.938477 14578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:45:12.939989 14578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:38715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:12.940277 14578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:12.940546 14578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:12.945463 14587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:12.945461 14592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:12.945463 14590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:12.945761 14578 server_base.cc:1034] running on GCE node
I20260805 22:45:12.946178 14578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:12.946493 14578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:12.947675 14578 hybrid_clock.cc:648] HybridClock initialized: now 1785969912947659 us; error 51 us; skew 500 ppm
Aug 05 22:45:12 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969912, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:12.950448 14578 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:45:12.950973 14578 webserver.cc:460] Webserver started at http://127.23.197.126:39085/ using document root <none> and password file <none>
I20260805 22:45:12.951176 14578 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:12.951222 14578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:12.951321 14578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:45:12.952448 14578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "79e060515c6d407cac5ae058b3edb973"
format_stamp: "Formatted at 2026-08-05 22:45:12 on dist-test-slave-g170"
server_key: "a8febda545fd6ea40c68380809c37755"
server_key_iv: "16b519e298d9741d7112dfc3430c249b"
server_key_version: "encryptionkey@0"
I20260805 22:45:12.952863 14578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "79e060515c6d407cac5ae058b3edb973"
format_stamp: "Formatted at 2026-08-05 22:45:12 on dist-test-slave-g170"
server_key: "a8febda545fd6ea40c68380809c37755"
server_key_iv: "16b519e298d9741d7112dfc3430c249b"
server_key_version: "encryptionkey@0"
I20260805 22:45:12.954293 14578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:45:12.955134 14603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:12.955348 14578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:45:12.955461 14578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "79e060515c6d407cac5ae058b3edb973"
format_stamp: "Formatted at 2026-08-05 22:45:12 on dist-test-slave-g170"
server_key: "a8febda545fd6ea40c68380809c37755"
server_key_iv: "16b519e298d9741d7112dfc3430c249b"
server_key_version: "encryptionkey@0"
I20260805 22:45:12.955538 14578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:12.969259 14578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:12.971632 14578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:12.971794 14578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:12.975894 14578 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:38715
I20260805 22:45:12.975965 14684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:38715 every 8 connection(s)
I20260805 22:45:12.976320 14578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:45:12.977360 14685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:45:12.978163 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 14578
I20260805 22:45:12.978286 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:45:12.978494 24341 external_mini_cluster.cc:1428] Setting key 82d4978f6fd7448e2642122223e95d7f
I20260805 22:45:12.980717 14685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Bootstrap starting.
I20260805 22:45:12.981393 14685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Neither blocks nor log segments found. Creating new log.
I20260805 22:45:12.981717 14685 log.cc:824] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:12.982470 14685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: No bootstrap required, opened a new log
I20260805 22:45:12.984370 14685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:12.984542 14685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:45:12.984594 14685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e060515c6d407cac5ae058b3edb973, State: Initialized, Role: FOLLOWER
I20260805 22:45:12.984827 14685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:12.984951 14685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:45:12.985001 14685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:45:12.985052 14685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 0 FOLLOWER]: Advancing to term 1
Aug 05 22:45:12 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:45:12.985743 14685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:12.985874 14685 leader_election.cc:302] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79e060515c6d407cac5ae058b3edb973; no voters:
I20260805 22:45:12.986120 14685 leader_election.cc:288] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:45:12.986160 14691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:45:12.986335 14691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 1 LEADER]: Becoming Leader. State: Replica: 79e060515c6d407cac5ae058b3edb973, State: Running, Role: LEADER
I20260805 22:45:12.986479 14685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:45:12.986480 14691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:12.986857 14691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79e060515c6d407cac5ae058b3edb973. Latest consensus state: current_term: 1 leader_uuid: "79e060515c6d407cac5ae058b3edb973" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } } }
I20260805 22:45:12.987016 14691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: This master's current role is: LEADER
I20260805 22:45:12.987362 14700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:45:12.986896 14693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79e060515c6d407cac5ae058b3edb973" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } } }
I20260805 22:45:12.987581 14693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: This master's current role is: LEADER
I20260805 22:45:12.988014 14700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:45:12.989843 14700 catalog_manager.cc:1347] Generated new cluster ID: 40698bc76bb2461bb18a0936b19c4377
I20260805 22:45:12.989918 14700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:45:12.990123 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:12.979079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48346 (local address 127.23.197.126:38715)
0805 22:45:12.979297 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:12.979307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:12.979341 (+ 34us) server_negotiation.cc:407] Connection header received
0805 22:45:12.979916 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:12.979926 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:12.980005 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:12.980193 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:12.980895 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:12.981894 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:12.982687 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:12.983114 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:12.983251 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:12.985829 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:12.985834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:12.985835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:12.985855 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:12.987984 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:12.988377 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:12.988381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:12.988381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:12.988499 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:12.988926 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:12.988927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:12.988928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:12.989206 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:12.989404 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:12.989778 (+ 374us) server_negotiation.cc:299] Negotiation successful
0805 22:45:12.989927 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260805 22:45:13.002660 14700 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:45:13.003162 14700 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:45:13.008286 14700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Generated new TSK 0
I20260805 22:45:13.008497 14700 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:45:13.040104 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38715
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:45:13.137123 14725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:13.137370 14725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:13.137398 14725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:13.138741 14725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:13.138787 14725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:13.138818 14725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:45:13.140468 14725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:13.140698 14725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:13.140995 14725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:13.147069 14737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:13.147069 14733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:13.147374 14745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:13.147704 14725 server_base.cc:1034] running on GCE node
I20260805 22:45:13.147976 14725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:13.148418 14725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:13.149600 14725 hybrid_clock.cc:648] HybridClock initialized: now 1785969913149590 us; error 46 us; skew 500 ppm
Aug 05 22:45:13 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:13.152570 14725 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:45:13.153206 14725 webserver.cc:460] Webserver started at http://127.23.197.65:44695/ using document root <none> and password file <none>
I20260805 22:45:13.153475 14725 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:13.153554 14725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:13.153725 14725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:45:13.154913 14725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "44f85fd1ba984682851b5602af408f52"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "ae55dd8c4c5b85690285fb51d4e6269d"
server_key_iv: "e7b13f6db02ee380b52b38929283210b"
server_key_version: "encryptionkey@0"
I20260805 22:45:13.155346 14725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "44f85fd1ba984682851b5602af408f52"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "ae55dd8c4c5b85690285fb51d4e6269d"
server_key_iv: "e7b13f6db02ee380b52b38929283210b"
server_key_version: "encryptionkey@0"
I20260805 22:45:13.157037 14725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:45:13.158057 14757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:13.158381 14725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:45:13.158511 14725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "44f85fd1ba984682851b5602af408f52"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "ae55dd8c4c5b85690285fb51d4e6269d"
server_key_iv: "e7b13f6db02ee380b52b38929283210b"
server_key_version: "encryptionkey@0"
I20260805 22:45:13.158607 14725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:13.177637 14725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:13.180205 14725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:13.180380 14725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:13.180665 14725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:45:13.181037 14725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:45:13.181115 14725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:13.181164 14725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:45:13.181219 14725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:13.187377 14725 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:33071
I20260805 22:45:13.187425 14931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:33071 every 8 connection(s)
I20260805 22:45:13.187798 14725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
Aug 05 22:45:13 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:45:13.197050 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 14725
I20260805 22:45:13.197191 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:45:13.197431 24341 external_mini_cluster.cc:1428] Setting key 847ff7a66671af4328afd17bfecc0cb7
I20260805 22:45:13.198024 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.188595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:44991 (local address 127.23.197.126:38715)
0805 22:45:13.188663 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.188665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.188958 (+ 293us) server_negotiation.cc:407] Connection header received
0805 22:45:13.189446 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.189447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.189459 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.189527 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:13.190496 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.191057 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.192101 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.192316 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.192377 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.194736 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:13.194737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:13.194738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:13.194745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:13.196267 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:13.196684 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:13.196685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:13.196685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:13.196848 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:13.197166 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:13.197167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:13.197168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:13.197367 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:13.197431 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.197822 (+ 391us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.197930 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 22:45:13.198201 14937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.188605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:44991)
0805 22:45:13.188807 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:13.188860 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:13.189337 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:13.189582 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:13.189586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:13.190176 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:45:13.190371 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.190375 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.191160 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.191161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.192004 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.192007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.192403 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.192404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.192502 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.193132 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:45:13.193140 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:45:13.194562 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:45:13.196351 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:13.196358 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:13.196360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:13.196577 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:13.196927 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:13.196928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:13.196929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:13.197082 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:13.197442 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:45:13.197450 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:45:13.197660 (+ 210us) client_negotiation.cc:769] Sending connection context
0805 22:45:13.197779 (+ 119us) client_negotiation.cc:240] Negotiation successful
0805 22:45:13.197951 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260805 22:45:13.198879 14933 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38715
I20260805 22:45:13.198961 14933 heartbeater.cc:464] Registering TS with master...
I20260805 22:45:13.199160 14933 heartbeater.cc:511] Master 127.23.197.126:38715 requested a full tablet report, sending...
I20260805 22:45:13.199764 14630 ts_manager.cc:194] Registered new tserver with Master: 44f85fd1ba984682851b5602af408f52 (127.23.197.65:33071)
I20260805 22:45:13.200380 14630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:44991
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:45:13.227392 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38715
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:45:13.310894 14940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:13.311131 14940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:13.311182 14940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:13.312307 14940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:13.312371 14940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:13.312415 14940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:45:13.313870 14940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:13.314131 14940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:13.314365 14940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:13.319821 14953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:13.319823 14952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:13.319794 14955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:13.320071 14940 server_base.cc:1034] running on GCE node
I20260805 22:45:13.320297 14940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:13.320590 14940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:13.321738 14940 hybrid_clock.cc:648] HybridClock initialized: now 1785969913321724 us; error 36 us; skew 500 ppm
Aug 05 22:45:13 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:13.324231 14940 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:45:13.324702 14940 webserver.cc:460] Webserver started at http://127.23.197.66:34691/ using document root <none> and password file <none>
I20260805 22:45:13.324972 14940 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:13.325021 14940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:13.325193 14940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:45:13.326296 14940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "d4664e528bd6480388fe7b4acb57ff3b"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "1dfe39496c3bd3f0f525af241607d296"
server_key_iv: "2c380058374978f85df003d32da0b087"
server_key_version: "encryptionkey@0"
I20260805 22:45:13.326670 14940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "d4664e528bd6480388fe7b4acb57ff3b"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "1dfe39496c3bd3f0f525af241607d296"
server_key_iv: "2c380058374978f85df003d32da0b087"
server_key_version: "encryptionkey@0"
I20260805 22:45:13.328042 14940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:45:13.328982 14965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:13.329177 14940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:45:13.329255 14940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "d4664e528bd6480388fe7b4acb57ff3b"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "1dfe39496c3bd3f0f525af241607d296"
server_key_iv: "2c380058374978f85df003d32da0b087"
server_key_version: "encryptionkey@0"
I20260805 22:45:13.329347 14940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:13.342577 14940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:13.345031 14940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:13.345170 14940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:13.345404 14940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:45:13.345741 14940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:45:13.345774 14940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:13.345798 14940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:45:13.345845 14940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:13.351905 14940 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:37151
I20260805 22:45:13.351948 15138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:37151 every 8 connection(s)
I20260805 22:45:13.352274 14940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:45:13.353266 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 14940
I20260805 22:45:13.353391 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:45:13.353644 24341 external_mini_cluster.cc:1428] Setting key 37d413634611f9dadf0f850e3c2df8bc
Aug 05 22:45:13 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
I20260805 22:45:13.364445 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.353035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:43973 (local address 127.23.197.126:38715)
0805 22:45:13.353112 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.353115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.353388 (+ 273us) server_negotiation.cc:407] Connection header received
0805 22:45:13.353909 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.353910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.353924 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.353966 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:13.354937 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.355445 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.356903 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.357096 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.357172 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.361010 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:13.361012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:13.361013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:13.361021 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:13.362718 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:13.363278 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:13.363279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:13.363280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:13.363352 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:13.363699 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:13.363700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:13.363701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:13.363836 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:13.363909 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.364189 (+ 280us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.364319 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 22:45:13.364502 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.353038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:43973)
0805 22:45:13.353243 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:13.353287 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:13.353793 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:13.354054 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:13.354058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:13.354648 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:45:13.354836 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.354841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.355543 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.355544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.356795 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.356799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.358240 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.358241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.358404 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.359043 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:45:13.359052 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:45:13.360905 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:45:13.362808 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:13.362815 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:13.362817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:13.363102 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:13.363433 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:13.363434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:13.363434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:13.363580 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:13.363913 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:45:13.363918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:45:13.364033 (+ 115us) client_negotiation.cc:769] Sending connection context
0805 22:45:13.364141 (+ 108us) client_negotiation.cc:240] Negotiation successful
0805 22:45:13.364310 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260805 22:45:13.365116 15139 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38715
I20260805 22:45:13.365288 15139 heartbeater.cc:464] Registering TS with master...
I20260805 22:45:13.365496 15139 heartbeater.cc:511] Master 127.23.197.126:38715 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
I20260805 22:45:13.365993 14630 ts_manager.cc:194] Registered new tserver with Master: d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
I20260805 22:45:13.366526 14630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:43973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:45:13.383358 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38715
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:45:13.472355 15148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:13.472617 15148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:13.472668 15148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:13.473968 15148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:13.474040 15148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:13.474083 15148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:45:13.475548 15148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:38715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:13.475834 15148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:13.476076 15148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:13.481606 15164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:13.481803 15167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:13.481741 15148 server_base.cc:1034] running on GCE node
W20260805 22:45:13.481608 15160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:13.482120 15148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:13.482425 15148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:13.483618 15148 hybrid_clock.cc:648] HybridClock initialized: now 1785969913483607 us; error 30 us; skew 500 ppm
Aug 05 22:45:13 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:13.486423 15148 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:45:13.486945 15148 webserver.cc:460] Webserver started at http://127.23.197.67:38193/ using document root <none> and password file <none>
I20260805 22:45:13.487293 15148 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:13.487381 15148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:13.487571 15148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:45:13.488981 15148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "41b80512e3c1c15e782a9b4a5ca52675"
server_key_iv: "14a3b5e245ba2a7eeb8f094fb854f634"
server_key_version: "encryptionkey@0"
I20260805 22:45:13.489434 15148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "41b80512e3c1c15e782a9b4a5ca52675"
server_key_iv: "14a3b5e245ba2a7eeb8f094fb854f634"
server_key_version: "encryptionkey@0"
I20260805 22:45:13.490916 15148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:45:13.491829 15179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:13.492067 15148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:45:13.492185 15148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "41b80512e3c1c15e782a9b4a5ca52675"
server_key_iv: "14a3b5e245ba2a7eeb8f094fb854f634"
server_key_version: "encryptionkey@0"
I20260805 22:45:13.492272 15148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:13.516347 15148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:13.518940 15148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:13.519147 15148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:13.519438 15148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:45:13.519788 15148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:45:13.519847 15148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:13.519893 15148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:45:13.519924 15148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:13.526345 15148 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:37693
I20260805 22:45:13.526391 15370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:37693 every 8 connection(s)
I20260805 22:45:13.526724 15148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:45:13.530098 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 15148
I20260805 22:45:13.530227 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:45:13.530433 24341 external_mini_cluster.cc:1428] Setting key 6b922f38c9ebeb745200b160768f0c5f
Aug 05 22:45:13 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:45:13.536546 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.527515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:37619 (local address 127.23.197.126:38715)
0805 22:45:13.527584 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.527586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.527820 (+ 234us) server_negotiation.cc:407] Connection header received
0805 22:45:13.528345 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.528347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.528358 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.528435 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:13.529424 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.529986 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.531229 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.531435 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.531525 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.533899 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:13.533901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:13.533901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:13.533909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:13.535115 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:13.535532 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:13.535533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:13.535534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:13.535597 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:13.535900 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:13.535901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:13.535902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:13.536017 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:13.536091 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.536379 (+ 288us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.536460 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260805 22:45:13.536749 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.527498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:37619)
0805 22:45:13.527678 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:13.527726 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:13.528233 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:13.528488 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:13.528492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:13.529103 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:45:13.529310 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.529315 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.530149 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.530151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.531106 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.531109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.531534 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.531535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.531673 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.532246 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:45:13.532255 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:45:13.533801 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:45:13.535222 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:13.535230 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:13.535232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:13.535451 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:13.535711 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:13.535712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:13.535712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:13.535822 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:13.536094 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:45:13.536101 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:45:13.536241 (+ 140us) client_negotiation.cc:769] Sending connection context
0805 22:45:13.536345 (+ 104us) client_negotiation.cc:240] Negotiation successful
0805 22:45:13.536500 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260805 22:45:13.537258 15371 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38715
I20260805 22:45:13.537335 15371 heartbeater.cc:464] Registering TS with master...
I20260805 22:45:13.537536 15371 heartbeater.cc:511] Master 127.23.197.126:38715 requested a full tablet report, sending...
I20260805 22:45:13.537911 14630 ts_manager.cc:194] Registered new tserver with Master: 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693)
I20260805 22:45:13.538244 14630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:37619
I20260805 22:45:13.542013 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:45:13.549423 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.543326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48358 (local address 127.23.197.126:38715)
0805 22:45:13.543418 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.543420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.543522 (+ 102us) server_negotiation.cc:407] Connection header received
0805 22:45:13.543583 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.543584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.543594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.543644 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:13.544189 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.544655 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.545429 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.545635 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.545720 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.546816 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:13.546817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:13.546818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:13.546825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:13.548147 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:13.548524 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:13.548525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:13.548525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:13.548584 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:13.548836 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:13.548837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:13.548838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:13.548994 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:13.549107 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.549238 (+ 131us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.549320 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 22:45:13.550992 14630 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:45:13.553010 14630 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:45:13.559782 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.555352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37151 (local address 127.0.0.1:43812)
0805 22:45:13.555528 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:13.555570 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:13.555639 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:13.556081 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:13.556082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:13.556084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:13.556183 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.556185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.557484 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.557485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.558581 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.558584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.559420 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.559421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.559517 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.559531 (+ 14us) client_negotiation.cc:769] Sending connection context
0805 22:45:13.559632 (+ 101us) client_negotiation.cc:240] Negotiation successful
0805 22:45:13.559717 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260805 22:45:13.559782 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.555568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.23.197.66:37151)
0805 22:45:13.555807 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.555812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.555822 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:45:13.555860 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.555862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.555879 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.555991 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:13.556277 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.557387 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.558653 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.559324 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.559420 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.559480 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.559651 (+ 171us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.559694 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":79,"threads_started":1}
I20260805 22:45:13.560887 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.555352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33071 (local address 127.0.0.1:33540)
0805 22:45:13.555820 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:13.555829 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:13.555874 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:13.556039 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:13.556044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:13.556046 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:13.556120 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.556122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.558666 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.558667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.559677 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.559680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.560632 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.560633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.560691 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.560694 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:45:13.560752 (+ 58us) client_negotiation.cc:240] Negotiation successful
0805 22:45:13.560815 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":4,"thread_start_us":49,"threads_started":1}
I20260805 22:45:13.560935 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.555559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33540 (local address 127.23.197.65:33071)
0805 22:45:13.555766 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.555771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.555880 (+ 109us) server_negotiation.cc:407] Connection header received
0805 22:45:13.555933 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.555935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.555950 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.556025 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:13.556203 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.557375 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.559773 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.560483 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.560549 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.560597 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.560809 (+ 212us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.560858 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
I20260805 22:45:13.560846 15034 tablet_service.cc:1467] Processing CreateTablet for tablet ff1782fdd1b34e1dbd987fb638ee016d (DEFAULT_TABLE table=test-table [id=c94b888a7ef8417d901c8a3dea90577d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:45:13.561277 15034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1782fdd1b34e1dbd987fb638ee016d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:45:13.561342 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.555358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:37693 (local address 127.0.0.1:52906)
0805 22:45:13.557514 (+ 2156us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:13.557525 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:13.557603 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:13.557788 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:13.557789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:13.557791 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:13.557859 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.557860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.558980 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.558981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.560094 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.560096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.561087 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.561089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.561176 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.561180 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:45:13.561238 (+ 58us) client_negotiation.cc:240] Negotiation successful
0805 22:45:13.561278 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2143,"thread_start_us":123,"threads_started":1}
I20260805 22:45:13.561403 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.555614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52906 (local address 127.23.197.67:37693)
0805 22:45:13.555801 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.555807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.557605 (+ 1798us) server_negotiation.cc:407] Connection header received
0805 22:45:13.557679 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.557681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.557696 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.557792 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:13.557913 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.558871 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.560183 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.560919 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.560989 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.561052 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.561279 (+ 227us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.561318 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":80,"threads_started":1}
I20260805 22:45:13.562077 15263 tablet_service.cc:1467] Processing CreateTablet for tablet ff1782fdd1b34e1dbd987fb638ee016d (DEFAULT_TABLE table=test-table [id=c94b888a7ef8417d901c8a3dea90577d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:45:13.562431 15263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1782fdd1b34e1dbd987fb638ee016d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:45:13.564976 15401 tablet_bootstrap.cc:492] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: Bootstrap starting.
I20260805 22:45:13.565361 14831 tablet_service.cc:1467] Processing CreateTablet for tablet ff1782fdd1b34e1dbd987fb638ee016d (DEFAULT_TABLE table=test-table [id=c94b888a7ef8417d901c8a3dea90577d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:45:13.565809 15401 tablet_bootstrap.cc:654] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: Neither blocks nor log segments found. Creating new log.
I20260805 22:45:13.565804 14831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1782fdd1b34e1dbd987fb638ee016d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:45:13.566303 15403 tablet_bootstrap.cc:492] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Bootstrap starting.
I20260805 22:45:13.566447 15401 log.cc:824] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:13.567183 15403 tablet_bootstrap.cc:654] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Neither blocks nor log segments found. Creating new log.
I20260805 22:45:13.567667 15403 log.cc:824] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:13.568398 15403 tablet_bootstrap.cc:492] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: No bootstrap required, opened a new log
I20260805 22:45:13.568485 15403 ts_tablet_manager.cc:1397] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:45:13.568969 15409 tablet_bootstrap.cc:492] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Bootstrap starting.
I20260805 22:45:13.569423 15401 tablet_bootstrap.cc:492] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: No bootstrap required, opened a new log
I20260805 22:45:13.569501 15401 ts_tablet_manager.cc:1397] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260805 22:45:13.569731 15409 tablet_bootstrap.cc:654] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Neither blocks nor log segments found. Creating new log.
I20260805 22:45:13.570230 15409 log.cc:824] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:13.570713 15403 raft_consensus.cc:348] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:13.570906 15403 raft_consensus.cc:374] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:45:13.570986 15403 raft_consensus.cc:729] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c1c7f8c13f43aa9ee4e19eea50c6ef, State: Initialized, Role: FOLLOWER
I20260805 22:45:13.571152 15403 consensus_queue.cc:260] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:13.571270 15409 tablet_bootstrap.cc:492] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: No bootstrap required, opened a new log
I20260805 22:45:13.571365 15409 ts_tablet_manager.cc:1397] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:45:13.571470 15403 ts_tablet_manager.cc:1428] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:45:13.571512 15371 heartbeater.cc:503] Master 127.23.197.126:38715 was elected leader, sending a full tablet report...
I20260805 22:45:13.571563 15401 raft_consensus.cc:348] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:13.571722 15401 raft_consensus.cc:374] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:45:13.571746 15401 raft_consensus.cc:729] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4664e528bd6480388fe7b4acb57ff3b, State: Initialized, Role: FOLLOWER
I20260805 22:45:13.571862 15401 consensus_queue.cc:260] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:13.572181 15401 ts_tablet_manager.cc:1428] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:45:13.572184 15139 heartbeater.cc:503] Master 127.23.197.126:38715 was elected leader, sending a full tablet report...
I20260805 22:45:13.573591 15409 raft_consensus.cc:348] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:13.573740 15409 raft_consensus.cc:374] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:45:13.573765 15409 raft_consensus.cc:729] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f85fd1ba984682851b5602af408f52, State: Initialized, Role: FOLLOWER
I20260805 22:45:13.573905 15409 consensus_queue.cc:260] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:13.574220 15409 ts_tablet_manager.cc:1428] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260805 22:45:13.574312 14933 heartbeater.cc:503] Master 127.23.197.126:38715 was elected leader, sending a full tablet report...
W20260805 22:45:13.602983 15143 tablet.cc:2369] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:45:13.688592 14936 tablet.cc:2369] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:45:13.692838 15412 raft_consensus.cc:482] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:45:13.692989 15412 raft_consensus.cc:504] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:13.693388 15412 leader_election.cc:288] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44f85fd1ba984682851b5602af408f52 (127.23.197.65:33071), d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
I20260805 22:45:13.696516 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.693698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33071 (local address 127.23.197.67:32991)
0805 22:45:13.693748 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:13.693762 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:13.693818 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:13.694017 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:13.694019 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:13.694021 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:13.694098 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.694100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.694744 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.694746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.695562 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.695565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.696195 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.696196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.696325 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.696329 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:45:13.696403 (+ 74us) client_negotiation.cc:240] Negotiation successful
0805 22:45:13.696442 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260805 22:45:13.696516 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.693717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:32991 (local address 127.23.197.65:33071)
0805 22:45:13.693781 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.693784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.693799 (+ 15us) server_negotiation.cc:407] Connection header received
0805 22:45:13.693891 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.693894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.693907 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.693993 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:13.694172 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.694653 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.695657 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.696097 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.696198 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.696209 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.696410 (+ 201us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.696453 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 22:45:13.696890 14866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1782fdd1b34e1dbd987fb638ee016d" candidate_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f85fd1ba984682851b5602af408f52" is_pre_election: true
I20260805 22:45:13.697088 14866 raft_consensus.cc:2454] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c1c7f8c13f43aa9ee4e19eea50c6ef in term 0.
I20260805 22:45:13.697273 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.693700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37151 (local address 127.23.197.67:49829)
0805 22:45:13.693978 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:13.693987 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:13.694073 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:13.694260 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:13.694261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:13.694263 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:13.694355 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.694362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.695213 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.695214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.696203 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:13.696206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.696977 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.696978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:13.697064 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.697069 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:45:13.697133 (+ 64us) client_negotiation.cc:240] Negotiation successful
0805 22:45:13.697178 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":60,"thread_start_us":147,"threads_started":1}
I20260805 22:45:13.697292 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.693807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:49829 (local address 127.23.197.66:37151)
0805 22:45:13.693852 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.693854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.694070 (+ 216us) server_negotiation.cc:407] Connection header received
0805 22:45:13.694174 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.694175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.694186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.694256 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:13.694442 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.695117 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.696284 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.696879 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.696978 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.696987 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.697153 (+ 166us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.697198 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:45:13.697371 15184 leader_election.cc:302] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44f85fd1ba984682851b5602af408f52, 97c1c7f8c13f43aa9ee4e19eea50c6ef; no voters:
I20260805 22:45:13.697530 15412 raft_consensus.cc:2781] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:45:13.697606 15412 raft_consensus.cc:482] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:45:13.697782 15066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1782fdd1b34e1dbd987fb638ee016d" candidate_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4664e528bd6480388fe7b4acb57ff3b" is_pre_election: true
I20260805 22:45:13.697923 15066 raft_consensus.cc:2454] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c1c7f8c13f43aa9ee4e19eea50c6ef in term 0.
I20260805 22:45:13.697642 15412 raft_consensus.cc:3037] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:45:13.698936 15412 raft_consensus.cc:504] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:13.699088 15412 leader_election.cc:288] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [CANDIDATE]: Term 1 election: Requested vote from peers 44f85fd1ba984682851b5602af408f52 (127.23.197.65:33071), d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
I20260805 22:45:13.699229 15066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1782fdd1b34e1dbd987fb638ee016d" candidate_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4664e528bd6480388fe7b4acb57ff3b"
I20260805 22:45:13.699283 14866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1782fdd1b34e1dbd987fb638ee016d" candidate_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f85fd1ba984682851b5602af408f52"
I20260805 22:45:13.699321 15066 raft_consensus.cc:3037] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:45:13.699357 14866 raft_consensus.cc:3037] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:45:13.700232 15066 raft_consensus.cc:2454] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c1c7f8c13f43aa9ee4e19eea50c6ef in term 1.
I20260805 22:45:13.700232 14866 raft_consensus.cc:2454] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c1c7f8c13f43aa9ee4e19eea50c6ef in term 1.
I20260805 22:45:13.700486 15180 leader_election.cc:302] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c1c7f8c13f43aa9ee4e19eea50c6ef, d4664e528bd6480388fe7b4acb57ff3b; no voters:
I20260805 22:45:13.700606 15412 raft_consensus.cc:2781] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:45:13.700750 15412 raft_consensus.cc:686] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 LEADER]: Becoming Leader. State: Replica: 97c1c7f8c13f43aa9ee4e19eea50c6ef, State: Running, Role: LEADER
I20260805 22:45:13.700932 15412 consensus_queue.cc:237] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:13.701845 14629 catalog_manager.cc:5495] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67). New cstate: current_term: 1 leader_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } health_report { overall_health: HEALTHY } } }
I20260805 22:45:13.741268 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.738722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52914 (local address 127.23.197.67:37693)
0805 22:45:13.738800 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:13.738803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:13.738812 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:45:13.738861 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:13.738862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:13.738881 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:13.738964 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:45:13.739117 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.739606 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.740396 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:13.740597 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:13.740675 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:13.740896 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:45:13.741005 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:45:13.741105 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:13.741145 (+ 40us) server_negotiation.cc:299] Negotiation successful
0805 22:45:13.741166 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 22:45:13.743337 15066 raft_consensus.cc:1261] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [term 1 FOLLOWER]: Refusing update from remote peer 97c1c7f8c13f43aa9ee4e19eea50c6ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:45:13.743376 14866 raft_consensus.cc:1261] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Refusing update from remote peer 97c1c7f8c13f43aa9ee4e19eea50c6ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:45:13.743875 15416 consensus_queue.cc:1035] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:45:13.744071 15412 consensus_queue.cc:1035] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:45:13.746966 15426 mvcc.cc:205] Tried to move back new op lower bound from 7315332766689615872 to 7315332766519676928. Current Snapshot: MvccSnapshot[applied={T|T < 7315332766689615872}]
I20260805 22:45:13.749716 14630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:45:13.749822 14630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:45:13.750648 14630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Sending DeleteTablet for 3 replicas of tablet ff1782fdd1b34e1dbd987fb638ee016d
I20260805 22:45:13.751026 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 14725
I20260805 22:45:13.751041 14831 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1782fdd1b34e1dbd987fb638ee016d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:45:13 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:33540
I20260805 22:45:13.751068 15034 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1782fdd1b34e1dbd987fb638ee016d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:45:13 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:43812
I20260805 22:45:13.757071 15263 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1782fdd1b34e1dbd987fb638ee016d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:45:13 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:52906
I20260805 22:45:13.757426 15438 tablet_replica.cc:330] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: stopping tablet replica
I20260805 22:45:13.757539 15438 raft_consensus.cc:2229] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:45:13.757639 15438 raft_consensus.cc:2258] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:45:13.758670 15438 ts_tablet_manager.cc:1905] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:45:13.760908 15438 ts_tablet_manager.cc:1918] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:45:13.761056 15438 log.cc:1196] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/ff1782fdd1b34e1dbd987fb638ee016d
I20260805 22:45:13.761476 15438 ts_tablet_manager.cc:1939] T ff1782fdd1b34e1dbd987fb638ee016d P d4664e528bd6480388fe7b4acb57ff3b: Deleting consensus metadata
I20260805 22:45:13.762056 15440 tablet_replica.cc:330] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: stopping tablet replica
I20260805 22:45:13.762252 15440 raft_consensus.cc:2229] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:45:13.762403 14605 catalog_manager.cc:4841] TS d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151): tablet ff1782fdd1b34e1dbd987fb638ee016d (table test-table [id=c94b888a7ef8417d901c8a3dea90577d]) successfully deleted
I20260805 22:45:13.762524 15440 raft_consensus.cc:2258] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:45:13.763705 15440 ts_tablet_manager.cc:1905] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:45:13.765576 15440 ts_tablet_manager.cc:1918] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:45:13.765719 15440 log.cc:1196] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/ff1782fdd1b34e1dbd987fb638ee016d
I20260805 22:45:13.766027 15440 ts_tablet_manager.cc:1939] T ff1782fdd1b34e1dbd987fb638ee016d P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Deleting consensus metadata
I20260805 22:45:13.766463 14609 catalog_manager.cc:4841] TS 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693): tablet ff1782fdd1b34e1dbd987fb638ee016d (table test-table [id=c94b888a7ef8417d901c8a3dea90577d]) successfully deleted
W20260805 22:45:13.772965 14607 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:33071 (error 108)
I20260805 22:45:13.773180 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 14940
I20260805 22:45:13.775035 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:13.773452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33071 (local address 127.0.0.1:33546)
0805 22:45:13.774929 (+ 1477us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:13.774992 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1447}
W20260805 22:45:13.776294 14607 catalog_manager.cc:4584] TS 44f85fd1ba984682851b5602af408f52 (127.23.197.65:33071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff1782fdd1b34e1dbd987fb638ee016d: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33071: connect: Connection refused (error 111)
I20260805 22:45:13.782596 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 15148
I20260805 22:45:13.795275 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 14578
I20260805 22:45:33.808640 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38715
--webserver_interface=127.23.197.126
--webserver_port=39085
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:38715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:45:33.893435 15448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:33.893647 15448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:33.893671 15448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:33.894843 15448 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260805 22:45:33.894888 15448 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260805 22:45:33.894902 15448 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260805 22:45:33.894914 15448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:45:33.894927 15448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:33.894940 15448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:33.894953 15448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:45:33.895011 15448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:45:33.896554 15448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:38715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=39085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:33.896852 15448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:33.897110 15448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:33.907449 15456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:33.907588 15459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:33.907509 15455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:33.907511 15448 server_base.cc:1034] running on GCE node
I20260805 22:45:33.908072 15448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:33.908459 15448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:33.909639 15448 hybrid_clock.cc:648] HybridClock initialized: now 1785969933909606 us; error 51 us; skew 500 ppm
Aug 05 22:45:33 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969933, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:33.918823 15448 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:45:33.919392 15448 webserver.cc:460] Webserver started at http://127.23.197.126:39085/ using document root <none> and password file <none>
I20260805 22:45:33.919677 15448 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:33.919776 15448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:33.921448 15448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:45:33.922380 15470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:33.922640 15448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:45:33.922773 15448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "79e060515c6d407cac5ae058b3edb973"
format_stamp: "Formatted at 2026-08-05 22:45:12 on dist-test-slave-g170"
server_key: "a8febda545fd6ea40c68380809c37755"
server_key_iv: "16b519e298d9741d7112dfc3430c249b"
server_key_version: "encryptionkey@0"
I20260805 22:45:33.923362 15448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:33.938930 15448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:33.941134 15448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:33.941313 15448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:33.945822 15448 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:38715
I20260805 22:45:33.945856 15540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:38715 every 8 connection(s)
I20260805 22:45:33.946264 15448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:45:33.947829 15543 sys_catalog.cc:263] Verifying existing consensus state
I20260805 22:45:33.948593 15543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Bootstrap starting.
I20260805 22:45:33.955267 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 15448
I20260805 22:45:33.955610 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:33071
--local_ip_for_outbound_sockets=127.23.197.65
--tserver_master_addrs=127.23.197.126:38715
--webserver_port=44695
--webserver_interface=127.23.197.65
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:45:33.958217 15543 log.cc:824] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:33.960352 15543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:45:33.960669 15543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Bootstrap complete.
I20260805 22:45:33.963089 15543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:33.963351 15543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e060515c6d407cac5ae058b3edb973, State: Initialized, Role: FOLLOWER
I20260805 22:45:33.963539 15543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:33.963620 15543 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:45:33.963680 15543 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:45:33.963732 15543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 1 FOLLOWER]: Advancing to term 2
I20260805 22:45:33.964833 15543 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:33.964962 15543 leader_election.cc:302] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79e060515c6d407cac5ae058b3edb973; no voters:
I20260805 22:45:33.965178 15543 leader_election.cc:288] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 22:45:33.965219 15550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 2 FOLLOWER]: Leader election won for term 2
I20260805 22:45:33.965337 15550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 2 LEADER]: Becoming Leader. State: Replica: 79e060515c6d407cac5ae058b3edb973, State: Running, Role: LEADER
I20260805 22:45:33.965485 15543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:45:33.965502 15550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:33.965914 15555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79e060515c6d407cac5ae058b3edb973. Latest consensus state: current_term: 2 leader_uuid: "79e060515c6d407cac5ae058b3edb973" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } } }
I20260805 22:45:33.966003 15555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: This master's current role is: LEADER
I20260805 22:45:33.966254 15554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "79e060515c6d407cac5ae058b3edb973" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } } }
I20260805 22:45:33.966364 15554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: This master's current role is: LEADER
I20260805 22:45:33.967458 15577 catalog_manager.cc:1259] Loaded cluster ID: 40698bc76bb2461bb18a0936b19c4377
I20260805 22:45:33.967509 15577 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: loading cluster ID for follower catalog manager: success
I20260805 22:45:33.968686 15577 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: acquiring CA information for follower catalog manager: success
I20260805 22:45:33.969162 15577 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 22:45:33.969240 15565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:45:33.969662 15565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:45:33.969749 15565 catalog_manager.cc:1259] Loaded cluster ID: 40698bc76bb2461bb18a0936b19c4377
I20260805 22:45:33.969801 15565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:45:33.969980 15565 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:45:33.970054 15565 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Loaded TSK: 0
I20260805 22:45:33.985926 15565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Generated new TSK 1
I20260805 22:45:33.986035 15565 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 22:45:34.043663 15548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:34.043895 15548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:34.043934 15548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:34.045154 15548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:34.045223 15548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:34.045270 15548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:45:34.046813 15548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:33071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=44695
--tserver_master_addrs=127.23.197.126:38715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:34.047098 15548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:34.047356 15548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:34.053014 15588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:34.053014 15587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:34.053282 15548 server_base.cc:1034] running on GCE node
W20260805 22:45:34.053128 15591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:34.053634 15548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:34.053948 15548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:34.055130 15548 hybrid_clock.cc:648] HybridClock initialized: now 1785969934055116 us; error 32 us; skew 500 ppm
Aug 05 22:45:34 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969934, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:34.057770 15548 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:45:34.058235 15548 webserver.cc:460] Webserver started at http://127.23.197.65:44695/ using document root <none> and password file <none>
I20260805 22:45:34.058491 15548 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:34.058583 15548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:34.060041 15548 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:45:34.060838 15605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:34.061049 15548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:45:34.061215 15548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "44f85fd1ba984682851b5602af408f52"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "ae55dd8c4c5b85690285fb51d4e6269d"
server_key_iv: "e7b13f6db02ee380b52b38929283210b"
server_key_version: "encryptionkey@0"
I20260805 22:45:34.061581 15548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:34.088186 15548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:34.090651 15548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:34.090857 15548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:34.091135 15548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:45:34.091706 15617 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 22:45:34.093268 15548 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 22:45:34.093322 15548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260805 22:45:34.093355 15548 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 22:45:34.094038 15548 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 22:45:34.094085 15548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260805 22:45:34.094131 15617 tablet_bootstrap.cc:492] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Bootstrap starting.
I20260805 22:45:34.102185 15548 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:33071
I20260805 22:45:34.102267 15783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:33071 every 8 connection(s)
I20260805 22:45:34.102694 15548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:45:34.105361 15617 log.cc:824] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:34.107400 15617 tablet_bootstrap.cc:492] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:45:34.107694 15617 tablet_bootstrap.cc:492] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Bootstrap complete.
I20260805 22:45:34.108634 15617 ts_tablet_manager.cc:1397] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.007s
I20260805 22:45:34.109583 15617 raft_consensus.cc:348] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
Aug 05 22:45:34 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969934, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:45:34.109828 15617 raft_consensus.cc:729] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f85fd1ba984682851b5602af408f52, State: Initialized, Role: FOLLOWER
I20260805 22:45:34.110001 15617 consensus_queue.cc:260] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } }
I20260805 22:45:34.110265 15617 ts_tablet_manager.cc:1428] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260805 22:45:34.111940 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 15548
I20260805 22:45:34.112236 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:37151
--local_ip_for_outbound_sockets=127.23.197.66
--tserver_master_addrs=127.23.197.126:38715
--webserver_port=34691
--webserver_interface=127.23.197.66
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:45:34.113692 15787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.103535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:57155)
0805 22:45:34.103743 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.103795 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.104309 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.104779 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.104784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.105087 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:45:34.105281 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.105285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.106438 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.106440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.107248 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.107250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.107989 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.107990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.108127 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.108728 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:45:34.108739 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:45:34.110248 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:45:34.111956 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:34.111963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:34.111966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:34.112253 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:34.112531 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:34.112532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:34.112532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:34.112644 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:34.113067 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:45:34.113074 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:45:34.113210 (+ 136us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.113328 (+ 118us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.113503 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260805 22:45:34.113692 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.103600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:57155 (local address 127.23.197.126:38715)
0805 22:45:34.103814 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.103822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.103890 (+ 68us) server_negotiation.cc:407] Connection header received
0805 22:45:34.104533 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.104539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.104586 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.104795 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:34.105409 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.106300 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.107349 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.107870 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.107994 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.110347 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:34.110352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:34.110352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:34.110365 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:34.111850 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.112342 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.112345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.112346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.112451 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.112729 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.112730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.112730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.112990 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:34.113088 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.113384 (+ 296us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.113512 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":102,"threads_started":1}
I20260805 22:45:34.114382 15785 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38715
I20260805 22:45:34.114502 15785 heartbeater.cc:464] Registering TS with master...
I20260805 22:45:34.114687 15785 heartbeater.cc:511] Master 127.23.197.126:38715 requested a full tablet report, sending...
I20260805 22:45:34.115455 15493 ts_manager.cc:194] Registered new tserver with Master: 44f85fd1ba984682851b5602af408f52 (127.23.197.65:33071)
I20260805 22:45:34.116559 15493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:57155
I20260805 22:45:34.117520 15785 heartbeater.cc:503] Master 127.23.197.126:38715 was elected leader, sending a full tablet report...
Aug 05 22:45:34 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969933, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.65@KRBTEST.COM
I20260805 22:45:34.125310 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.116121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52584 (local address 127.23.197.65:33071)
0805 22:45:34.116271 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.116275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.116458 (+ 183us) server_negotiation.cc:407] Connection header received
0805 22:45:34.116528 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.116530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.116545 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.116619 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:34.117144 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.118448 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.119206 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.119707 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.119800 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.122195 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:34.122203 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:34.122203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:34.122210 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:34.123861 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.124362 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.124365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.124365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.124421 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.124658 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.124659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.124659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.124865 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:34.124934 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.125155 (+ 221us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.125241 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260805 22:45:34.125406 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.116052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33071 (local address 127.0.0.1:52584)
0805 22:45:34.116338 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.116373 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.116446 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.116632 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.116636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.116986 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:45:34.117055 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.117057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.118536 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.118537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.119116 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.119118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.119788 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.119789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.119845 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.120326 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:45:34.120331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:45:34.122108 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:45:34.123954 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:34.123963 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:34.123966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:34.124270 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:34.124500 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:34.124501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:34.124502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:34.124582 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:34.124937 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:45:34.124945 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:45:34.125012 (+ 67us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.125122 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.125283 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":91,"threads_started":1}
I20260805 22:45:34.125893 15673 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1782fdd1b34e1dbd987fb638ee016d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:45:13 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:52584
I20260805 22:45:34.126538 15800 tablet_replica.cc:330] stopping tablet replica
I20260805 22:45:34.126668 15800 raft_consensus.cc:2229] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:45:34.126797 15800 raft_consensus.cc:2258] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:45:34.127153 15800 ts_tablet_manager.cc:1905] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:45:34.129220 15800 ts_tablet_manager.cc:1918] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:45:34.129289 15800 log.cc:1196] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal/wals/ff1782fdd1b34e1dbd987fb638ee016d
I20260805 22:45:34.129532 15800 ts_tablet_manager.cc:1939] T ff1782fdd1b34e1dbd987fb638ee016d P 44f85fd1ba984682851b5602af408f52: Deleting consensus metadata
I20260805 22:45:34.130069 15474 catalog_manager.cc:4841] TS 44f85fd1ba984682851b5602af408f52 (127.23.197.65:33071): tablet ff1782fdd1b34e1dbd987fb638ee016d (table test-table [id=c94b888a7ef8417d901c8a3dea90577d]) successfully deleted
W20260805 22:45:34.200582 15794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:34.200871 15794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:34.200915 15794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:34.202087 15794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:34.202159 15794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:34.202206 15794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:45:34.203698 15794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:37151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=34691
--tserver_master_addrs=127.23.197.126:38715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:34.203997 15794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:34.204244 15794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:34.209708 15810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:34.209712 15813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:34.209769 15809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:34.209985 15794 server_base.cc:1034] running on GCE node
I20260805 22:45:34.210183 15794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:34.210445 15794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:34.211632 15794 hybrid_clock.cc:648] HybridClock initialized: now 1785969934211594 us; error 60 us; skew 500 ppm
Aug 05 22:45:34 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969934, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:34.214493 15794 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:45:34.215075 15794 webserver.cc:460] Webserver started at http://127.23.197.66:34691/ using document root <none> and password file <none>
I20260805 22:45:34.215356 15794 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:34.215453 15794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:34.217111 15794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:45:34.217875 15821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:34.218083 15794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:45:34.218194 15794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "d4664e528bd6480388fe7b4acb57ff3b"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "1dfe39496c3bd3f0f525af241607d296"
server_key_iv: "2c380058374978f85df003d32da0b087"
server_key_version: "encryptionkey@0"
I20260805 22:45:34.218581 15794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:34.226961 15794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:34.230041 15794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:34.230219 15794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:34.230471 15794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:45:34.230824 15794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:45:34.230885 15794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:34.230929 15794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:45:34.230957 15794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:34.236975 15794 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:37151
I20260805 22:45:34.237051 15993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:37151 every 8 connection(s)
I20260805 22:45:34.237452 15794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:45:34.239151 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 15794
I20260805 22:45:34.239471 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:37693
--local_ip_for_outbound_sockets=127.23.197.67
--tserver_master_addrs=127.23.197.126:38715
--webserver_port=38193
--webserver_interface=127.23.197.67
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 22:45:34 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969934, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:45:34.248339 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.238385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:48955 (local address 127.23.197.126:38715)
0805 22:45:34.238459 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.238461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.238770 (+ 309us) server_negotiation.cc:407] Connection header received
0805 22:45:34.239278 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.239279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.239292 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.239352 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:34.240545 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.241130 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.242323 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.242551 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.242637 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.245136 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:34.245138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:34.245139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:34.245149 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:34.246738 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.247237 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.247239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.247240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.247309 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.247602 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.247603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.247604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.247740 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:34.247822 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.248131 (+ 309us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.248238 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 22:45:34.248569 15998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.238389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:48955)
0805 22:45:34.238578 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.238616 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.239147 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.239433 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.239438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.240164 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:45:34.240418 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.240425 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.241231 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.241232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.242229 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.242231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.242646 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.242647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.242786 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.243369 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:45:34.243379 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:45:34.245015 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:45:34.246858 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:34.246869 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:34.246872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:34.247142 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:34.247390 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:34.247391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:34.247391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:34.247520 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:34.247811 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:45:34.247819 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:45:34.247983 (+ 164us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.248107 (+ 124us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.248301 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260805 22:45:34.249231 15994 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38715
I20260805 22:45:34.249325 15994 heartbeater.cc:464] Registering TS with master...
I20260805 22:45:34.249536 15994 heartbeater.cc:511] Master 127.23.197.126:38715 requested a full tablet report, sending...
I20260805 22:45:34.250029 15493 ts_manager.cc:194] Registered new tserver with Master: d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
I20260805 22:45:34.250437 15493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:48955
W20260805 22:45:34.332109 16003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:34.332350 16003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:34.332389 16003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:34.333572 16003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:34.333644 16003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:34.333686 16003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:45:34.335161 16003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:37693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=38193
--tserver_master_addrs=127.23.197.126:38715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:34.335446 16003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:34.335693 16003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 22:45:34.341001 16003 server_base.cc:1034] running on GCE node
W20260805 22:45:34.340970 16010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:34.340970 16013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:34.340956 16011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:34.341421 16003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:34.341681 16003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:34.342824 16003 hybrid_clock.cc:648] HybridClock initialized: now 1785969934342812 us; error 32 us; skew 500 ppm
Aug 05 22:45:34 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969934, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:34.345528 16003 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:45:34.345990 16003 webserver.cc:460] Webserver started at http://127.23.197.67:38193/ using document root <none> and password file <none>
I20260805 22:45:34.346251 16003 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:34.346340 16003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:34.347752 16003 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:45:34.348481 16021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:34.348935 16003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:45:34.349052 16003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "41b80512e3c1c15e782a9b4a5ca52675"
server_key_iv: "14a3b5e245ba2a7eeb8f094fb854f634"
server_key_version: "encryptionkey@0"
I20260805 22:45:34.349449 16003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:34.361274 16003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:34.363941 16003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:34.364120 16003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:34.364383 16003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:45:34.364833 16003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:45:34.364916 16003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:34.364969 16003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:45:34.365005 16003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:34.372124 16003 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:37693
I20260805 22:45:34.372164 16183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:37693 every 8 connection(s)
I20260805 22:45:34.372573 16003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:45:34.377517 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 16003
Aug 05 22:45:34 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969934, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:45:34.384377 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.373452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:52931 (local address 127.23.197.126:38715)
0805 22:45:34.373552 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.373555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.373878 (+ 323us) server_negotiation.cc:407] Connection header received
0805 22:45:34.374368 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.374369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.374383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.374476 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:34.375511 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.376095 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.377348 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.377592 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.377670 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.380904 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:34.380906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:34.380906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:34.380913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:34.382462 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.382990 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.382992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.382992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.383092 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.383393 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.383394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.383394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.383546 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:34.383626 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.384159 (+ 533us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.384248 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 22:45:34.384491 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.373457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:52931)
0805 22:45:34.373763 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.373799 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.374265 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.374598 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.374602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.375208 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:45:34.375404 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.375409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.376220 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.376221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.377239 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.377242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.377761 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.377763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.377949 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.378690 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:45:34.378702 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:45:34.380800 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:45:34.382554 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:34.382563 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:34.382566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:34.382881 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:34.383180 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:34.383181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:34.383181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:34.383318 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:34.383639 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:45:34.383646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:45:34.383950 (+ 304us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.384099 (+ 149us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.384271 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":87,"threads_started":1}
I20260805 22:45:34.385078 16184 heartbeater.cc:347] Connected to a master server at 127.23.197.126:38715
I20260805 22:45:34.385223 16184 heartbeater.cc:464] Registering TS with master...
I20260805 22:45:34.385473 16184 heartbeater.cc:511] Master 127.23.197.126:38715 requested a full tablet report, sending...
I20260805 22:45:34.385953 15493 ts_manager.cc:194] Registered new tserver with Master: 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693)
I20260805 22:45:34.386108 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.378031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.23.197.126:38715)
0805 22:45:34.378135 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.378137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.378340 (+ 203us) server_negotiation.cc:407] Connection header received
0805 22:45:34.378368 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.378369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.378383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.378455 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:34.379355 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.380072 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.381198 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.381420 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.381510 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.382994 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:34.382995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:34.382996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:34.383002 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:34.384804 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.385176 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.385178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.385178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.385237 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.385493 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.385493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.385494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.385642 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:34.385838 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.385909 (+ 71us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.385986 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":49,"threads_started":1}
I20260805 22:45:34.386467 15493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:52931
I20260805 22:45:34.386898 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:45:34.389542 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.387258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50954 (local address 127.23.197.126:38715)
0805 22:45:34.387298 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.387300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.387447 (+ 147us) server_negotiation.cc:407] Connection header received
0805 22:45:34.387507 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.387508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.387535 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.387591 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:45:34.387665 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.388118 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.388944 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.389129 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.389188 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.389363 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:45:34.389401 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0805 22:45:34.389503 (+ 102us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50954: token expired
Metrics: {"server-negotiator.queue_time_us":20}
W20260805 22:45:34.389633 16191 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50954: token expired
W20260805 22:45:34.389802 15377 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.23.197.126:38715)
I20260805 22:45:34.395947 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.390155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.23.197.126:38715)
0805 22:45:34.390222 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.390224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.390232 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:45:34.390252 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.390252 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.390260 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.390318 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:34.390786 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.391211 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.391957 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.392177 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.392269 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.393468 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:34.393469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:34.393469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:34.393475 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:34.394826 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.395166 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.395167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.395167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.395224 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:34.395403 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:34.395404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:34.395404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:34.395502 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:34.395593 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.395711 (+ 118us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.395837 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 22:45:34.397285 15493 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:45:34.398921 15493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:45:34.401229 15673 tablet_service.cc:1467] Processing CreateTablet for tablet df348033326c4d8fb784d8fb56b00ff7 (DEFAULT_TABLE table=test-table [id=d736f387aeb149aa857d8589ad749d0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:45:34.401467 15673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df348033326c4d8fb784d8fb56b00ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:45:34.402935 15617 tablet_bootstrap.cc:492] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Bootstrap starting.
I20260805 22:45:34.403306 15617 tablet_bootstrap.cc:654] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Neither blocks nor log segments found. Creating new log.
I20260805 22:45:34.403964 15617 tablet_bootstrap.cc:492] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: No bootstrap required, opened a new log
I20260805 22:45:34.404009 15617 ts_tablet_manager.cc:1397] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260805 22:45:34.404367 15617 raft_consensus.cc:348] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.404452 15617 raft_consensus.cc:374] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:45:34.404510 15617 raft_consensus.cc:729] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f85fd1ba984682851b5602af408f52, State: Initialized, Role: FOLLOWER
I20260805 22:45:34.404572 15617 consensus_queue.cc:260] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.404677 15617 ts_tablet_manager.cc:1428] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 22:45:34.405512 16194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.400978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37151 (local address 127.0.0.1:33880)
0805 22:45:34.401156 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.401167 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.401207 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.401726 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.401727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.401729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:34.401782 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.401784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.402865 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.402867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.404089 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.404092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.405230 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.405232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.405305 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.405336 (+ 31us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.405391 (+ 55us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.405439 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":123,"threads_started":1}
I20260805 22:45:34.405529 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.401254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33880 (local address 127.23.197.66:37151)
0805 22:45:34.401475 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.401480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.401498 (+ 18us) server_negotiation.cc:407] Connection header received
0805 22:45:34.401565 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.401567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.401584 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.401704 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:34.401862 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.402770 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.404192 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.405056 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.405121 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.405180 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.405408 (+ 228us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.405445 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":106,"threads_started":1}
I20260805 22:45:34.406175 15792 raft_consensus.cc:482] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:45:34.406268 15792 raft_consensus.cc:504] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.406436 15883 tablet_service.cc:1467] Processing CreateTablet for tablet df348033326c4d8fb784d8fb56b00ff7 (DEFAULT_TABLE table=test-table [id=d736f387aeb149aa857d8589ad749d0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:45:34.406566 15792 leader_election.cc:288] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693), d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
I20260805 22:45:34.406781 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.401044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51390 (local address 127.23.197.67:37693)
0805 22:45:34.403034 (+ 1990us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.403039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.403052 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:45:34.403100 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.403102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.403124 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.403218 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:34.403391 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.404461 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.405841 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.406410 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.406505 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.406565 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.406660 (+ 95us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.406705 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1947,"thread_start_us":116,"threads_started":1}
I20260805 22:45:34.406764 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.400938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:37693 (local address 127.0.0.1:51390)
0805 22:45:34.400990 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.401002 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.401041 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.403262 (+ 2221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.403263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.403265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:34.403307 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.403309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.404586 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.404587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.405654 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.405658 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.406504 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.406505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.406578 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.406582 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.406646 (+ 64us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.406691 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":2}
I20260805 22:45:34.406839 15883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df348033326c4d8fb784d8fb56b00ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:45:34.407560 16085 tablet_service.cc:1467] Processing CreateTablet for tablet df348033326c4d8fb784d8fb56b00ff7 (DEFAULT_TABLE table=test-table [id=d736f387aeb149aa857d8589ad749d0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:45:34.407918 16085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df348033326c4d8fb784d8fb56b00ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:45:34.409938 16202 tablet_bootstrap.cc:492] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: Bootstrap starting.
I20260805 22:45:34.410604 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.406917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:60403 (local address 127.23.197.67:37693)
0805 22:45:34.407053 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.407055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.407210 (+ 155us) server_negotiation.cc:407] Connection header received
0805 22:45:34.407334 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.407335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.407344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.407370 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:34.407584 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.408240 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.409658 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.410177 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.410228 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.410236 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.410508 (+ 272us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.410543 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":99,"threads_started":1}
I20260805 22:45:34.410898 16201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.406765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:37693 (local address 127.23.197.65:60403)
0805 22:45:34.407149 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.407160 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.407245 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.407432 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.407433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.407434 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:34.407495 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.407497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.408344 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.408345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.409552 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.409555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.410326 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.410327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.410399 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.410422 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.410729 (+ 307us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.410790 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359}
I20260805 22:45:34.411232 16202 tablet_bootstrap.cc:654] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: Neither blocks nor log segments found. Creating new log.
I20260805 22:45:34.411875 16202 log.cc:824] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:34.411259 16120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df348033326c4d8fb784d8fb56b00ff7" candidate_uuid: "44f85fd1ba984682851b5602af408f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" is_pre_election: true
I20260805 22:45:34.412304 16205 tablet_bootstrap.cc:492] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Bootstrap starting.
W20260805 22:45:34.412353 15613 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693): Illegal state: must be running to vote when last-logged opid is not known
I20260805 22:45:34.413266 16205 tablet_bootstrap.cc:654] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Neither blocks nor log segments found. Creating new log.
I20260805 22:45:34.413386 15787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.407004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37151 (local address 127.23.197.65:60111)
0805 22:45:34.410035 (+ 3031us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.410047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.410094 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.410255 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.410256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.410257 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:34.410316 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.410318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.411186 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.411187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.412297 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.412300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.413096 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.413097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.413210 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.413214 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.413276 (+ 62us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.413311 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3000,"thread_start_us":83,"threads_started":1}
I20260805 22:45:34.413731 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.407175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:60111 (local address 127.23.197.66:37151)
0805 22:45:34.407504 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.407506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.410146 (+ 2640us) server_negotiation.cc:407] Connection header received
0805 22:45:34.410168 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.410169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.410177 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.410219 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:34.410395 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.411074 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.412400 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.412992 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.413097 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.413106 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.413635 (+ 529us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.413675 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308}
I20260805 22:45:34.413729 16205 log.cc:824] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:34.414157 16202 tablet_bootstrap.cc:492] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: No bootstrap required, opened a new log
I20260805 22:45:34.414232 16202 ts_tablet_manager.cc:1397] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 22:45:34.414486 16205 tablet_bootstrap.cc:492] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: No bootstrap required, opened a new log
I20260805 22:45:34.414572 16205 ts_tablet_manager.cc:1397] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:45:34.413954 15919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df348033326c4d8fb784d8fb56b00ff7" candidate_uuid: "44f85fd1ba984682851b5602af408f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4664e528bd6480388fe7b4acb57ff3b" is_pre_election: true
W20260805 22:45:34.415222 15606 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151): Illegal state: must be running to vote when last-logged opid is not known
I20260805 22:45:34.415278 15606 leader_election.cc:302] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44f85fd1ba984682851b5602af408f52; no voters: 97c1c7f8c13f43aa9ee4e19eea50c6ef, d4664e528bd6480388fe7b4acb57ff3b
I20260805 22:45:34.415442 15792 raft_consensus.cc:2726] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260805 22:45:34.415808 16205 raft_consensus.cc:348] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.415822 16202 raft_consensus.cc:348] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.415934 16202 raft_consensus.cc:374] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:45:34.415937 16205 raft_consensus.cc:374] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:45:34.415956 16202 raft_consensus.cc:729] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4664e528bd6480388fe7b4acb57ff3b, State: Initialized, Role: FOLLOWER
I20260805 22:45:34.415980 16205 raft_consensus.cc:729] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c1c7f8c13f43aa9ee4e19eea50c6ef, State: Initialized, Role: FOLLOWER
I20260805 22:45:34.416038 16202 consensus_queue.cc:260] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.416098 16205 consensus_queue.cc:260] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.416344 16202 ts_tablet_manager.cc:1428] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:45:34.416364 15994 heartbeater.cc:503] Master 127.23.197.126:38715 was elected leader, sending a full tablet report...
I20260805 22:45:34.416375 16205 ts_tablet_manager.cc:1428] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:45:34.416529 16184 heartbeater.cc:503] Master 127.23.197.126:38715 was elected leader, sending a full tablet report...
W20260805 22:45:34.488132 15997 tablet.cc:2369] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:45:34.506767 16212 raft_consensus.cc:482] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:45:34.506963 16212 raft_consensus.cc:504] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.507373 16212 leader_election.cc:288] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151), 44f85fd1ba984682851b5602af408f52 (127.23.197.65:33071)
I20260805 22:45:34.511775 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.507882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:49747 (local address 127.23.197.66:37151)
0805 22:45:34.507930 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.507932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.507940 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:45:34.508098 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.508100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.508114 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.508195 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:34.508355 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.509139 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.510686 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.511278 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.511369 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.511379 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.511652 (+ 273us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.511702 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260805 22:45:34.511833 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.507670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:46489 (local address 127.23.197.65:33071)
0805 22:45:34.507722 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.507724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.507763 (+ 39us) server_negotiation.cc:407] Connection header received
0805 22:45:34.507809 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.507810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.507820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.507874 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:45:34.508120 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.509092 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.510686 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.511406 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.511475 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.511536 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.511661 (+ 125us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.511752 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:45:34.511801 16213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.507697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37151 (local address 127.23.197.67:49747)
0805 22:45:34.507893 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.507903 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.507947 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.508215 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.508215 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.508217 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:34.508280 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.508281 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.509244 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.509244 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.510573 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.510576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.511409 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.511410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.511529 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.511534 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.511607 (+ 73us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.511658 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":45,"thread_start_us":114,"threads_started":1}
I20260805 22:45:34.511801 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.507590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33071 (local address 127.23.197.67:46489)
0805 22:45:34.507651 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.507662 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:34.507723 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:34.507900 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:34.507902 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:34.507904 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:45:34.507985 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.507988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.509215 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.509217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.510573 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:34.510576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.511507 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.511508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:34.511583 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.511586 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:45:34.511640 (+ 54us) client_negotiation.cc:240] Negotiation successful
0805 22:45:34.511667 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":81}
I20260805 22:45:34.512158 15919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df348033326c4d8fb784d8fb56b00ff7" candidate_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4664e528bd6480388fe7b4acb57ff3b" is_pre_election: true
I20260805 22:45:34.512326 15919 raft_consensus.cc:2454] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c1c7f8c13f43aa9ee4e19eea50c6ef in term 0.
I20260805 22:45:34.512427 15708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df348033326c4d8fb784d8fb56b00ff7" candidate_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f85fd1ba984682851b5602af408f52" is_pre_election: true
I20260805 22:45:34.512568 15708 raft_consensus.cc:2454] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c1c7f8c13f43aa9ee4e19eea50c6ef in term 0.
I20260805 22:45:34.512545 16022 leader_election.cc:302] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c1c7f8c13f43aa9ee4e19eea50c6ef, d4664e528bd6480388fe7b4acb57ff3b; no voters:
I20260805 22:45:34.512770 16212 raft_consensus.cc:2781] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:45:34.512876 16212 raft_consensus.cc:482] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:45:34.512929 16212 raft_consensus.cc:3037] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:45:34.513662 16212 raft_consensus.cc:504] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.513823 16212 leader_election.cc:288] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [CANDIDATE]: Term 1 election: Requested vote from peers d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151), 44f85fd1ba984682851b5602af408f52 (127.23.197.65:33071)
I20260805 22:45:34.513995 15708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df348033326c4d8fb784d8fb56b00ff7" candidate_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44f85fd1ba984682851b5602af408f52"
I20260805 22:45:34.514091 15708 raft_consensus.cc:3037] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:45:34.514767 15708 raft_consensus.cc:2454] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c1c7f8c13f43aa9ee4e19eea50c6ef in term 1.
I20260805 22:45:34.514896 15919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df348033326c4d8fb784d8fb56b00ff7" candidate_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4664e528bd6480388fe7b4acb57ff3b"
I20260805 22:45:34.514946 16024 leader_election.cc:302] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44f85fd1ba984682851b5602af408f52, 97c1c7f8c13f43aa9ee4e19eea50c6ef; no voters:
I20260805 22:45:34.514983 15919 raft_consensus.cc:3037] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:45:34.515064 16212 raft_consensus.cc:2781] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:45:34.515229 16212 raft_consensus.cc:686] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 LEADER]: Becoming Leader. State: Replica: 97c1c7f8c13f43aa9ee4e19eea50c6ef, State: Running, Role: LEADER
I20260805 22:45:34.515348 16212 consensus_queue.cc:237] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:34.515892 15919 raft_consensus.cc:2454] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c1c7f8c13f43aa9ee4e19eea50c6ef in term 1.
I20260805 22:45:34.516309 15492 catalog_manager.cc:5495] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67). New cstate: current_term: 1 leader_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } health_report { overall_health: UNKNOWN } } }
I20260805 22:45:34.528699 16200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.526282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.23.197.67:37693)
0805 22:45:34.526336 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:34.526338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:34.526369 (+ 31us) server_negotiation.cc:407] Connection header received
0805 22:45:34.526459 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:34.526459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:34.526479 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:34.526558 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:45:34.526669 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.527301 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.528053 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:34.528259 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:34.528314 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:34.528444 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:45:34.528536 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:45:34.528590 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:34.528621 (+ 31us) server_negotiation.cc:299] Negotiation successful
0805 22:45:34.528638 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:45:34.530503 15919 raft_consensus.cc:1261] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [term 1 FOLLOWER]: Refusing update from remote peer 97c1c7f8c13f43aa9ee4e19eea50c6ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:45:34.530515 15708 raft_consensus.cc:1261] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Refusing update from remote peer 97c1c7f8c13f43aa9ee4e19eea50c6ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:45:34.530867 16217 consensus_queue.cc:1035] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:45:34.530990 16212 consensus_queue.cc:1035] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:45:34.533401 16221 mvcc.cc:205] Tried to move back new op lower bound from 7315332851834445824 to 7315332851775610880. Current Snapshot: MvccSnapshot[applied={T|T < 7315332851834445824}]
I20260805 22:45:34.535907 15493 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:45:34.535995 15493 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:45:34.536754 15493 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Sending DeleteTablet for 3 replicas of tablet df348033326c4d8fb784d8fb56b00ff7
I20260805 22:45:34.537089 16085 tablet_service.cc:1514] Processing DeleteTablet for tablet df348033326c4d8fb784d8fb56b00ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:45:34 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:51390
I20260805 22:45:34.537128 15673 tablet_service.cc:1514] Processing DeleteTablet for tablet df348033326c4d8fb784d8fb56b00ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:45:34 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:52584
I20260805 22:45:34.537236 15800 tablet_replica.cc:330] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: stopping tablet replica
I20260805 22:45:34.537227 15883 tablet_service.cc:1514] Processing DeleteTablet for tablet df348033326c4d8fb784d8fb56b00ff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:45:34 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:33880
I20260805 22:45:34.537299 15800 raft_consensus.cc:2229] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:45:34.537348 15800 raft_consensus.cc:2258] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:45:34.537501 16238 tablet_replica.cc:330] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: stopping tablet replica
I20260805 22:45:34.537608 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 15548
I20260805 22:45:34.537647 16238 raft_consensus.cc:2229] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:45:34.537693 15800 ts_tablet_manager.cc:1905] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:45:34.537847 16238 raft_consensus.cc:2258] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:45:34.544639 16234 tablet_replica.cc:330] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: stopping tablet replica
I20260805 22:45:34.544782 16234 raft_consensus.cc:2229] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:45:34.544802 16238 ts_tablet_manager.cc:1905] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:45:34.544920 16234 raft_consensus.cc:2258] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:45:34.545368 16234 ts_tablet_manager.cc:1905] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:45:34.546775 16238 ts_tablet_manager.cc:1918] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:45:34.546928 16238 log.cc:1196] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/df348033326c4d8fb784d8fb56b00ff7
I20260805 22:45:34.547111 16234 ts_tablet_manager.cc:1918] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:45:34.547217 16234 log.cc:1196] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/df348033326c4d8fb784d8fb56b00ff7
I20260805 22:45:34.547473 16238 ts_tablet_manager.cc:1939] T df348033326c4d8fb784d8fb56b00ff7 P d4664e528bd6480388fe7b4acb57ff3b: Deleting consensus metadata
I20260805 22:45:34.547598 16234 ts_tablet_manager.cc:1939] T df348033326c4d8fb784d8fb56b00ff7 P 97c1c7f8c13f43aa9ee4e19eea50c6ef: Deleting consensus metadata
I20260805 22:45:34.548127 15472 catalog_manager.cc:4841] TS d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151): tablet df348033326c4d8fb784d8fb56b00ff7 (table test-table [id=d736f387aeb149aa857d8589ad749d0c]) successfully deleted
I20260805 22:45:34.548130 15476 catalog_manager.cc:4841] TS 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693): tablet df348033326c4d8fb784d8fb56b00ff7 (table test-table [id=d736f387aeb149aa857d8589ad749d0c]) successfully deleted
W20260805 22:45:34.552199 15474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:33071 (error 108)
I20260805 22:45:34.552520 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 15794
I20260805 22:45:34.554275 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:34.552679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33071 (local address 127.0.0.1:52592)
0805 22:45:34.554179 (+ 1500us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:34.554236 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1474}
W20260805 22:45:34.554417 15474 catalog_manager.cc:4584] TS 44f85fd1ba984682851b5602af408f52 (127.23.197.65:33071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df348033326c4d8fb784d8fb56b00ff7: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33071: connect: Connection refused (error 111)
I20260805 22:45:34.559782 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 16003
I20260805 22:45:34.569656 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 15448
I20260805 22:45:54.580430 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38715
--webserver_interface=127.23.197.126
--webserver_port=39085
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:38715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:45:54.665243 16241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:54.665441 16241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:54.665462 16241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:54.666523 16241 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260805 22:45:54.666556 16241 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260805 22:45:54.666569 16241 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260805 22:45:54.666580 16241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:45:54.666594 16241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:54.666606 16241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:54.666618 16241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:45:54.666630 16241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:45:54.668061 16241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:38715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:38715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=39085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:54.668263 16241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:54.668430 16241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 22:45:54.673635 16241 server_base.cc:1034] running on GCE node
W20260805 22:45:54.673604 16249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:54.673641 16247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:54.673605 16246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:54.674207 16241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:54.674499 16241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:54.675705 16241 hybrid_clock.cc:648] HybridClock initialized: now 1785969954675692 us; error 32 us; skew 500 ppm
Aug 05 22:45:54 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969954, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:54.682464 16241 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:45:54.682981 16241 webserver.cc:460] Webserver started at http://127.23.197.126:39085/ using document root <none> and password file <none>
I20260805 22:45:54.683235 16241 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:54.683312 16241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:54.684684 16241 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:45:54.685501 16259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:54.685763 16241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:45:54.685879 16241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "79e060515c6d407cac5ae058b3edb973"
format_stamp: "Formatted at 2026-08-05 22:45:12 on dist-test-slave-g170"
server_key: "a8febda545fd6ea40c68380809c37755"
server_key_iv: "16b519e298d9741d7112dfc3430c249b"
server_key_version: "encryptionkey@0"
I20260805 22:45:54.686230 16241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:54.696890 16241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:54.699038 16241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:54.699210 16241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:54.703604 16241 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:38715
I20260805 22:45:54.703639 16346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:38715 every 8 connection(s)
I20260805 22:45:54.704032 16241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:45:54.705610 16347 sys_catalog.cc:263] Verifying existing consensus state
I20260805 22:45:54.706290 16347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Bootstrap starting.
I20260805 22:45:54.707134 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 16241
I20260805 22:45:54.707418 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:33071
--local_ip_for_outbound_sockets=127.23.197.65
--tserver_master_addrs=127.23.197.126:38715
--webserver_port=44695
--webserver_interface=127.23.197.65
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:45:54.718199 16347 log.cc:824] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:54.721952 16347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:45:54.722326 16347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Bootstrap complete.
I20260805 22:45:54.725752 16347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:54.726080 16347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e060515c6d407cac5ae058b3edb973, State: Initialized, Role: FOLLOWER
I20260805 22:45:54.726238 16347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:54.726306 16347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:45:54.726346 16347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:45:54.726392 16347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 2 FOLLOWER]: Advancing to term 3
I20260805 22:45:54.727698 16347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:54.727826 16347 leader_election.cc:302] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79e060515c6d407cac5ae058b3edb973; no voters:
I20260805 22:45:54.728003 16347 leader_election.cc:288] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260805 22:45:54.728127 16353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 3 FOLLOWER]: Leader election won for term 3
I20260805 22:45:54.728257 16347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:45:54.728339 16353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [term 3 LEADER]: Becoming Leader. State: Replica: 79e060515c6d407cac5ae058b3edb973, State: Running, Role: LEADER
I20260805 22:45:54.728480 16353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } }
I20260805 22:45:54.728888 16354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "79e060515c6d407cac5ae058b3edb973" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } } }
I20260805 22:45:54.728969 16354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: This master's current role is: LEADER
I20260805 22:45:54.729382 16355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79e060515c6d407cac5ae058b3edb973. Latest consensus state: current_term: 3 leader_uuid: "79e060515c6d407cac5ae058b3edb973" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e060515c6d407cac5ae058b3edb973" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 38715 } } }
I20260805 22:45:54.729460 16355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973 [sys.catalog]: This master's current role is: LEADER
I20260805 22:45:54.729645 16365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:45:54.730432 16365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:45:54.730562 16365 catalog_manager.cc:1259] Loaded cluster ID: 40698bc76bb2461bb18a0936b19c4377
I20260805 22:45:54.730623 16365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:45:54.731403 16365 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:45:54.731590 16365 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Loaded TSK: 0
I20260805 22:45:54.731647 16365 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79e060515c6d407cac5ae058b3edb973: Loaded TSK: 1
F20260805 22:45:54.731858 16365 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f86974c89ad google::LogMessage::Fail() at ??:0
@ 0x7f86974cb88a google::LogMessage::SendToLog() at ??:0
@ 0x7f86974c84de google::LogMessage::Flush() at ??:0
@ 0x7f86974c9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8699c35104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8699c32d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f869f4221cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f869f42248e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f869f422594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f869f3e8c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f869f3f2ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f86993960d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f869938b954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f869a1ef6db start_thread at ??:0
@ 0x7f8695fc661f clone at ??:0
W20260805 22:45:54.812021 16351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:54.812254 16351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:54.812294 16351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:54.813527 16351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:54.813592 16351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:54.813635 16351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:45:54.815115 16351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:33071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=44695
--tserver_master_addrs=127.23.197.126:38715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:54.815402 16351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:54.815641 16351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:54.821753 16417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:54.821782 16415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:54.821965 16351 server_base.cc:1034] running on GCE node
W20260805 22:45:54.821758 16414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:54.822292 16351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:54.822582 16351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:54.823760 16351 hybrid_clock.cc:648] HybridClock initialized: now 1785969954823744 us; error 32 us; skew 500 ppm
Aug 05 22:45:54 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969954, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:54.826352 16351 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:45:54.826892 16351 webserver.cc:460] Webserver started at http://127.23.197.65:44695/ using document root <none> and password file <none>
I20260805 22:45:54.827142 16351 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:54.827188 16351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:54.828625 16351 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:45:54.829488 16428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:54.829674 16351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:45:54.829782 16351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "44f85fd1ba984682851b5602af408f52"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "ae55dd8c4c5b85690285fb51d4e6269d"
server_key_iv: "e7b13f6db02ee380b52b38929283210b"
server_key_version: "encryptionkey@0"
I20260805 22:45:54.830143 16351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:54.841259 16351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:54.843842 16351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:54.843995 16351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:54.844290 16351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:45:54.844821 16442 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 22:45:54.846325 16351 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 22:45:54.846379 16351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260805 22:45:54.846415 16351 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 22:45:54.847092 16351 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 22:45:54.847141 16351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260805 22:45:54.847191 16442 tablet_bootstrap.cc:492] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Bootstrap starting.
I20260805 22:45:54.849416 16442 log.cc:824] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Log is configured to *not* fsync() on all Append() calls
I20260805 22:45:54.852957 16442 tablet_bootstrap.cc:492] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 22:45:54.853291 16442 tablet_bootstrap.cc:492] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Bootstrap complete.
I20260805 22:45:54.853529 16442 ts_tablet_manager.cc:1397] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260805 22:45:54.854923 16351 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:33071
I20260805 22:45:54.855001 16605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:33071 every 8 connection(s)
I20260805 22:45:54.855384 16351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:45:54.855775 16442 raft_consensus.cc:348] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:54.856288 16442 raft_consensus.cc:729] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f85fd1ba984682851b5602af408f52, State: Initialized, Role: FOLLOWER
I20260805 22:45:54.856441 16442 consensus_queue.cc:260] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:54.856791 16442 ts_tablet_manager.cc:1428] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260805 22:45:54.864570 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 16351
I20260805 22:45:54.864938 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:37151
--local_ip_for_outbound_sockets=127.23.197.66
--tserver_master_addrs=127.23.197.126:38715
--webserver_port=34691
--webserver_interface=127.23.197.66
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:45:54.936371 16608 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:45:54.856091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:48441)
0805 22:45:54.856379 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:54.856423 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:54.856589 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:54.936145 (+ 79556us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":105,"threads_started":1}
W20260805 22:45:54.936597 16606 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:38715 (0 consecutive failures): Network error: Failed to ping master at 127.23.197.126:38715: Client connection negotiation failed: client connection to 127.23.197.126:38715: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 22:45:54.937003 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:54.936878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:45555)
0805 22:45:54.936931 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:54.936977 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260805 22:45:54.937463 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:54.937323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:59745)
0805 22:45:54.937400 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:54.937436 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260805 22:45:54.937659 16606 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 22:45:54.953799 16610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:54.954032 16610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:54.954073 16610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:54.955292 16610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:54.955332 16610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:54.955360 16610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:45:54.958186 16610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:37151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=34691
--tserver_master_addrs=127.23.197.126:38715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:54.958483 16610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:54.958745 16610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:54.964316 16623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:54.964349 16621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:54.964331 16620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:54.964743 16610 server_base.cc:1034] running on GCE node
I20260805 22:45:54.965036 16610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:54.965328 16610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:54.966483 16610 hybrid_clock.cc:648] HybridClock initialized: now 1785969954966463 us; error 39 us; skew 500 ppm
Aug 05 22:45:54 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969954, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:54.969251 16610 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:45:54.969746 16610 webserver.cc:460] Webserver started at http://127.23.197.66:34691/ using document root <none> and password file <none>
I20260805 22:45:54.969967 16610 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:54.970011 16610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:54.971446 16610 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260805 22:45:54.972149 16633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:54.972354 16610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:45:54.972447 16610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "d4664e528bd6480388fe7b4acb57ff3b"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "1dfe39496c3bd3f0f525af241607d296"
server_key_iv: "2c380058374978f85df003d32da0b087"
server_key_version: "encryptionkey@0"
I20260805 22:45:54.972867 16610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:54.989711 16610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:54.992151 16610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:54.992278 16610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:54.992478 16610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:45:54.992880 16610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:45:54.992923 16610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:54.992946 16610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:45:54.992988 16610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:54.999147 16610 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:37151
I20260805 22:45:54.999257 16799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:37151 every 8 connection(s)
I20260805 22:45:54.999584 16610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:45:55.000708 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:55.000326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:47003)
0805 22:45:55.000487 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:55.000560 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
W20260805 22:45:55.001212 16800 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:38715 (0 consecutive failures): Network error: Failed to ping master at 127.23.197.126:38715: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
I20260805 22:45:55.001580 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:55.001472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:56687)
0805 22:45:55.001523 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:55.001557 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260805 22:45:55.001994 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:55.001910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:40711)
0805 22:45:55.001950 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:55.001976 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260805 22:45:55.002162 16800 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 22:45:55.002800 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 16610
I20260805 22:45:55.003082 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:37693
--local_ip_for_outbound_sockets=127.23.197.67
--tserver_master_addrs=127.23.197.126:38715
--webserver_port=38193
--webserver_interface=127.23.197.67
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:45:55.098482 16805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:45:55.098708 16805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:45:55.098747 16805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:45:55.100052 16805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:45:55.100100 16805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:45:55.100129 16805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:45:55.101784 16805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:37693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=38193
--tserver_master_addrs=127.23.197.126:38715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:45:55.101999 16805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:45:55.102206 16805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:45:55.107414 16813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:55.107455 16816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:45:55.107405 16811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:45:55.107848 16805 server_base.cc:1034] running on GCE node
I20260805 22:45:55.108099 16805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:45:55.108315 16805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:45:55.109488 16805 hybrid_clock.cc:648] HybridClock initialized: now 1785969955109476 us; error 37 us; skew 500 ppm
Aug 05 22:45:55 dist-test-slave-g170 krb5kdc[14561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969955, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:45:55.112251 16805 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:45:55.112845 16805 webserver.cc:460] Webserver started at http://127.23.197.67:38193/ using document root <none> and password file <none>
I20260805 22:45:55.113121 16805 fs_manager.cc:359] Metadata directory not provided
I20260805 22:45:55.113201 16805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:45:55.114773 16805 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:45:55.115425 16824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:55.115643 16805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:45:55.115789 16805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef"
format_stamp: "Formatted at 2026-08-05 22:45:13 on dist-test-slave-g170"
server_key: "41b80512e3c1c15e782a9b4a5ca52675"
server_key_iv: "14a3b5e245ba2a7eeb8f094fb854f634"
server_key_version: "encryptionkey@0"
I20260805 22:45:55.116173 16805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:45:55.127091 16805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:45:55.129590 16805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:45:55.129791 16805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:45:55.130061 16805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:45:55.130412 16805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:45:55.130473 16805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:55.130511 16805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:45:55.130558 16805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:45:55.137598 16805 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:37693
I20260805 22:45:55.137652 17000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:37693 every 8 connection(s)
I20260805 22:45:55.138047 16805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:45:55.139211 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:55.138809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:45737)
0805 22:45:55.138975 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:55.139055 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260805 22:45:55.139748 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 16805
W20260805 22:45:55.139776 17001 heartbeater.cc:644] Failed to heartbeat to 127.23.197.126:38715 (0 consecutive failures): Network error: Failed to ping master at 127.23.197.126:38715: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
I20260805 22:45:55.140100 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:55.140003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:42049)
0805 22:45:55.140042 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:55.140069 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260805 22:45:55.140442 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:55.140313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:43071)
0805 22:45:55.140352 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:55.140405 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260805 22:45:55.140607 17001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 22:45:55.938392 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:55.938121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:47373)
0805 22:45:55.938269 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:55.938310 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260805 22:45:56.002830 17009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:56.002575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:56647)
0805 22:45:56.002734 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:56.002776 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260805 22:45:56.012956 17011 raft_consensus.cc:482] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:45:56.013164 17011 raft_consensus.cc:504] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:56.013764 17011 leader_election.cc:288] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693), d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
Aug 05 22:45:56 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969954, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.66@KRBTEST.COM
Aug 05 22:45:56 dist-test-slave-g170 krb5kdc[14561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969954, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.67@KRBTEST.COM
I20260805 22:45:56.026626 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:56.013937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:37693 (local address 127.23.197.65:33123)
0805 22:45:56.014004 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:56.014016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:56.014080 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:56.014982 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:56.014986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:56.015320 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:45:56.015536 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:56.015542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:56.016696 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:56.016697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:56.017527 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:56.017530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:56.018244 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:56.018246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:56.018436 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:56.019331 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:45:56.019338 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:45:56.021599 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:45:56.024129 (+ 2530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:56.024144 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:56.024144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:56.024743 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:56.025113 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:56.025113 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:56.025114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:56.025284 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:56.025915 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:45:56.025926 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:45:56.026130 (+ 204us) client_negotiation.cc:769] Sending connection context
0805 22:45:56.026251 (+ 121us) client_negotiation.cc:240] Negotiation successful
0805 22:45:56.026480 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":31}
I20260805 22:45:56.026629 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:56.014087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:33123 (local address 127.23.197.67:37693)
0805 22:45:56.014289 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:56.014297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:56.014317 (+ 20us) server_negotiation.cc:407] Connection header received
0805 22:45:56.014732 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:56.014734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:56.014795 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:56.014953 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:56.015673 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:56.016578 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:56.017621 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:56.018135 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:56.018263 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:56.021718 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:56.021723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:56.021723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:56.021737 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:56.024023 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:56.024876 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:56.024880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:56.024880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:56.025019 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:56.025416 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:56.025417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:56.025418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:56.025807 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:56.025934 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:56.026375 (+ 441us) server_negotiation.cc:299] Negotiation successful
0805 22:45:56.026527 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
I20260805 22:45:56.026643 17016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:56.014050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:55713 (local address 127.23.197.66:37151)
0805 22:45:56.014230 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0805 22:45:56.014245 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 22:45:56.014292 (+ 47us) server_negotiation.cc:407] Connection header received
0805 22:45:56.014872 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:45:56.014874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:45:56.014916 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:45:56.015107 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:45:56.015886 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:56.016880 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:56.017675 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:45:56.018351 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:45:56.018485 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:56.021392 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:45:56.021397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:45:56.021398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:45:56.021417 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:45:56.023969 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:56.024876 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:56.024880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:56.024880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:56.025019 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:45:56.025396 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:45:56.025398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:45:56.025398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:45:56.025830 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:45:56.025920 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 22:45:56.026371 (+ 451us) server_negotiation.cc:299] Negotiation successful
0805 22:45:56.026525 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":57,"threads_started":1}
I20260805 22:45:56.026706 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:56.013938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37151 (local address 127.23.197.65:55713)
0805 22:45:56.014068 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:56.014079 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:45:56.014135 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:45:56.015086 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:45:56.015087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:45:56.015328 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:45:56.015740 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:56.015743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:56.016989 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:56.016990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:56.017567 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:45:56.017568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:45:56.018454 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:45:56.018455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:45:56.018534 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:45:56.019291 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:45:56.019319 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:45:56.021260 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:45:56.024129 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:56.024139 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:56.024142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:56.024743 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:56.025113 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:45:56.025113 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:45:56.025114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:45:56.025284 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:45:56.025915 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:45:56.025926 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:45:56.026130 (+ 204us) client_negotiation.cc:769] Sending connection context
0805 22:45:56.026251 (+ 121us) client_negotiation.cc:240] Negotiation successful
0805 22:45:56.026546 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":172,"thread_start_us":54,"threads_started":1}
W20260805 22:45:56.028453 16431 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
W20260805 22:45:56.028574 16437 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
I20260805 22:45:56.028632 16437 leader_election.cc:302] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44f85fd1ba984682851b5602af408f52; no voters: 97c1c7f8c13f43aa9ee4e19eea50c6ef, d4664e528bd6480388fe7b4acb57ff3b
I20260805 22:45:56.028744 17011 raft_consensus.cc:2726] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:45:56.141227 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:56.141010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:48689)
0805 22:45:56.141133 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:56.141182 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260805 22:45:56.939213 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:56.938971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:49181)
0805 22:45:56.939123 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:56.939164 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260805 22:45:57.003618 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:57.003407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:53593)
0805 22:45:57.003534 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:57.003575 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260805 22:45:57.142007 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:57.141745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:43771)
0805 22:45:57.141922 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:57.141964 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
I20260805 22:45:57.940030 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:57.939763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:43155)
0805 22:45:57.939926 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:57.939972 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260805 22:45:58.004601 17024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:58.004353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:45261)
0805 22:45:58.004509 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:58.004554 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260805 22:45:58.142910 17026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:58.142566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:45141)
0805 22:45:58.142791 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:58.142837 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":93,"threads_started":1}
I20260805 22:45:58.209751 17027 raft_consensus.cc:482] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:45:58.209853 17027 raft_consensus.cc:504] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:45:58.210106 17027 leader_election.cc:288] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693), d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
W20260805 22:45:58.210559 16431 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
W20260805 22:45:58.210637 16437 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
I20260805 22:45:58.210714 16431 leader_election.cc:302] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44f85fd1ba984682851b5602af408f52; no voters: 97c1c7f8c13f43aa9ee4e19eea50c6ef, d4664e528bd6480388fe7b4acb57ff3b
I20260805 22:45:58.210799 17027 raft_consensus.cc:2726] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:45:58.941004 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:58.940684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:58855)
0805 22:45:58.940893 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:58.940950 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":94,"threads_started":1}
I20260805 22:45:59.005389 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:59.005167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:36241)
0805 22:45:59.005311 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:59.005344 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260805 22:45:59.143863 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:59.143622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:52311)
0805 22:45:59.143781 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:59.143817 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260805 22:45:59.942170 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:45:59.941863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:32799)
0805 22:45:59.942057 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0805 22:45:59.942110 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260805 22:46:00.006249 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:00.006002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:42829)
0805 22:46:00.006143 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:00.006201 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260805 22:46:00.144719 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:00.144454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:43721)
0805 22:46:00.144616 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:00.144668 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260805 22:46:00.809044 17039 raft_consensus.cc:482] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:46:00.809157 17039 raft_consensus.cc:504] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:46:00.809372 17039 leader_election.cc:288] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693), d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
W20260805 22:46:00.809818 16437 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
W20260805 22:46:00.809914 16431 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
I20260805 22:46:00.809996 16431 leader_election.cc:302] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44f85fd1ba984682851b5602af408f52; no voters: 97c1c7f8c13f43aa9ee4e19eea50c6ef, d4664e528bd6480388fe7b4acb57ff3b
I20260805 22:46:00.810055 17039 raft_consensus.cc:2726] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:46:00.943101 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:00.942794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:55755)
0805 22:46:00.942975 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:00.943041 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260805 22:46:01.007143 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:01.006842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:53711)
0805 22:46:01.007047 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:01.007093 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260805 22:46:01.145628 17045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:01.145352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:50777)
0805 22:46:01.145501 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:01.145560 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260805 22:46:01.943859 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:01.943583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:40203)
0805 22:46:01.943774 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:01.943812 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":126,"threads_started":1}
I20260805 22:46:02.008019 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:02.007755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:54233)
0805 22:46:02.007928 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:02.007970 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":67,"threads_started":1}
I20260805 22:46:02.146493 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:02.146238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:42369)
0805 22:46:02.146406 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:02.146448 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260805 22:46:02.944533 17052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:02.944293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:52231)
0805 22:46:02.944442 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:02.944481 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260805 22:46:03.008881 17054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:03.008564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:52547)
0805 22:46:03.008723 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:03.008821 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260805 22:46:03.147393 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:03.147089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:40609)
0805 22:46:03.147299 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:03.147343 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":115,"threads_started":1}
I20260805 22:46:03.473305 17056 raft_consensus.cc:482] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:46:03.473412 17056 raft_consensus.cc:504] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:46:03.473601 17056 leader_election.cc:288] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693), d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
W20260805 22:46:03.473994 16431 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
W20260805 22:46:03.474097 16437 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
I20260805 22:46:03.474175 16437 leader_election.cc:302] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44f85fd1ba984682851b5602af408f52; no voters: 97c1c7f8c13f43aa9ee4e19eea50c6ef, d4664e528bd6480388fe7b4acb57ff3b
I20260805 22:46:03.474270 17056 raft_consensus.cc:2726] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:46:03.945499 17058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:03.945214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:53003)
0805 22:46:03.945380 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:03.945446 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260805 22:46:04.009748 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:04.009437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:52847)
0805 22:46:04.009646 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:04.009697 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":68,"threads_started":1}
I20260805 22:46:04.148203 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:04.147938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:44515)
0805 22:46:04.148108 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:04.148155 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":83,"threads_started":1}
I20260805 22:46:04.946280 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:04.946034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:41577)
0805 22:46:04.946191 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:04.946231 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260805 22:46:05.010653 17062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:05.010433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:56371)
0805 22:46:05.010565 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:05.010605 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260805 22:46:05.149037 17063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:05.148747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:40787)
0805 22:46:05.148950 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:05.148990 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":131,"threads_started":1}
I20260805 22:46:05.947012 17064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:05.946780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:45797)
0805 22:46:05.946922 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:05.946963 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260805 22:46:06.011579 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:06.011326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:33543)
0805 22:46:06.011482 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:06.011530 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260805 22:46:06.113592 17068 raft_consensus.cc:482] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:46:06.113720 17068 raft_consensus.cc:504] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c1c7f8c13f43aa9ee4e19eea50c6ef" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 37693 } } peers { permanent_uuid: "d4664e528bd6480388fe7b4acb57ff3b" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37151 } } peers { permanent_uuid: "44f85fd1ba984682851b5602af408f52" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33071 } }
I20260805 22:46:06.113940 17068 leader_election.cc:288] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693), d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151)
W20260805 22:46:06.114413 16437 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97c1c7f8c13f43aa9ee4e19eea50c6ef (127.23.197.67:37693): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
W20260805 22:46:06.114511 16431 leader_election.cc:341] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4664e528bd6480388fe7b4acb57ff3b (127.23.197.66:37151): Not found: Tablet not found: df348033326c4d8fb784d8fb56b00ff7
I20260805 22:46:06.114563 16431 leader_election.cc:302] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44f85fd1ba984682851b5602af408f52; no voters: 97c1c7f8c13f43aa9ee4e19eea50c6ef, d4664e528bd6480388fe7b4acb57ff3b
I20260805 22:46:06.114678 17068 raft_consensus.cc:2726] T df348033326c4d8fb784d8fb56b00ff7 P 44f85fd1ba984682851b5602af408f52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 22:46:06.149864 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:06.149567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:45319)
0805 22:46:06.149770 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:06.149815 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":87,"threads_started":1}
I20260805 22:46:06.947932 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:06.947609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:45997)
0805 22:46:06.947831 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:06.947877 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":136,"threads_started":1}
I20260805 22:46:07.012575 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:07.012291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:52785)
0805 22:46:07.012474 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:07.012524 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":82,"threads_started":1}
I20260805 22:46:07.150683 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:07.150413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:42421)
0805 22:46:07.150580 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:07.150632 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
I20260805 22:46:07.948887 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:07.948544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:59271)
0805 22:46:07.948788 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:07.948835 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":74,"threads_started":1}
I20260805 22:46:08.013442 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:08.013182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:33927)
0805 22:46:08.013340 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:08.013389 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260805 22:46:08.151473 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:08.151208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:39695)
0805 22:46:08.151375 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:08.151424 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":85,"threads_started":1}
I20260805 22:46:08.949846 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:08.949584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:37887)
0805 22:46:08.949748 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:08.949794 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260805 22:46:09.014343 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:09.014094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:39043)
0805 22:46:09.014249 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:09.014291 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260805 22:46:09.152313 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:09.152053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:48517)
0805 22:46:09.152227 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:09.152265 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260805 22:46:09.950763 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:09.950505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.65:51407)
0805 22:46:09.950672 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:09.950716 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260805 22:46:10.015123 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:10.014917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.66:32935)
0805 22:46:10.015039 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:10.015075 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260805 22:46:10.147567 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 16351
I20260805 22:46:10.154122 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:10.153211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:38715 (local address 127.23.197.67:36835)
0805 22:46:10.154012 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:10.154051 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.126:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":703,"threads_started":1}
I20260805 22:46:10.162803 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 16610
I20260805 22:46:10.171618 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 16805
2026-08-05T22:46:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58719 ms)
[----------] 30 tests from SecurityITest (245092 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:10 dist-test-slave-g170 krb5kdc[17090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:10 dist-test-slave-g170 krb5kdc[17090](info): set up 2 sockets
Aug 05 22:46:10 dist-test-slave-g170 krb5kdc[17090](info): commencing operation
krb5kdc: starting...
W20260805 22:46:11.434320 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:11 dist-test-slave-g170 krb5kdc[17090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:11.567478 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:32885
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:32885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:11.650781 17106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:11.651008 17106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:11.651047 17106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:11.652192 17106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:11.652259 17106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:11.652285 17106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:11.652316 17106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:11.652341 17106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:11.653901 17106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:32885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:32885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:11.654186 17106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:11.654417 17106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:11.659480 17117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:11.659512 17121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:11.659533 17106 server_base.cc:1034] running on GCE node
W20260805 22:46:11.659508 17118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:11.660090 17106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:11.660399 17106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:11.661643 17106 hybrid_clock.cc:648] HybridClock initialized: now 1785969971661572 us; error 47 us; skew 500 ppm
Aug 05 22:46:11 dist-test-slave-g170 krb5kdc[17090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969971, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:11.669303 17106 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:11.669867 17106 webserver.cc:460] Webserver started at http://127.23.197.126:45707/ using document root <none> and password file <none>
I20260805 22:46:11.670116 17106 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:11.670188 17106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:11.670363 17106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:11.671484 17106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "5d9e778ae5d448229ec5b655ec13a9a6"
format_stamp: "Formatted at 2026-08-05 22:46:11 on dist-test-slave-g170"
server_key: "fe44d66b8befd89d87c42556c9a940e0"
server_key_iv: "03c69e4bc46ddf492c3354473a4f2f4f"
server_key_version: "encryptionkey@0"
I20260805 22:46:11.671887 17106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "5d9e778ae5d448229ec5b655ec13a9a6"
format_stamp: "Formatted at 2026-08-05 22:46:11 on dist-test-slave-g170"
server_key: "fe44d66b8befd89d87c42556c9a940e0"
server_key_iv: "03c69e4bc46ddf492c3354473a4f2f4f"
server_key_version: "encryptionkey@0"
I20260805 22:46:11.673430 17106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:11.674369 17130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:11.674595 17106 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:11.674754 17106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "5d9e778ae5d448229ec5b655ec13a9a6"
format_stamp: "Formatted at 2026-08-05 22:46:11 on dist-test-slave-g170"
server_key: "fe44d66b8befd89d87c42556c9a940e0"
server_key_iv: "03c69e4bc46ddf492c3354473a4f2f4f"
server_key_version: "encryptionkey@0"
I20260805 22:46:11.674845 17106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:11.683130 17106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:11.685428 17106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:11.685598 17106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:11.690111 17106 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:32885
I20260805 22:46:11.690150 17215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:32885 every 8 connection(s)
I20260805 22:46:11.690541 17106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:11.691416 17216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:11.693073 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 17106
I20260805 22:46:11.693217 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:11.693413 24341 external_mini_cluster.cc:1428] Setting key d46efc41a1c5f2b7adee0f7ce3836aca
I20260805 22:46:11.694293 17216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6: Bootstrap starting.
I20260805 22:46:11.694983 17216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:11.695343 17216 log.cc:824] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:11.696197 17216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6: No bootstrap required, opened a new log
I20260805 22:46:11.698921 17216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9e778ae5d448229ec5b655ec13a9a6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 32885 } }
I20260805 22:46:11.699107 17216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:11.699153 17216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d9e778ae5d448229ec5b655ec13a9a6, State: Initialized, Role: FOLLOWER
I20260805 22:46:11.699297 17216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9e778ae5d448229ec5b655ec13a9a6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 32885 } }
I20260805 22:46:11.699385 17216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:11.699432 17216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:11.699484 17216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:11.700310 17216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9e778ae5d448229ec5b655ec13a9a6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 32885 } }
I20260805 22:46:11.700408 17216 leader_election.cc:302] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d9e778ae5d448229ec5b655ec13a9a6; no voters:
I20260805 22:46:11.700609 17216 leader_election.cc:288] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:11.700634 17228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:11.700800 17228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [term 1 LEADER]: Becoming Leader. State: Replica: 5d9e778ae5d448229ec5b655ec13a9a6, State: Running, Role: LEADER
Aug 05 22:46:11 dist-test-slave-g170 krb5kdc[17090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:11.700968 17216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:11.700950 17228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9e778ae5d448229ec5b655ec13a9a6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 32885 } }
I20260805 22:46:11.701296 17228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d9e778ae5d448229ec5b655ec13a9a6. Latest consensus state: current_term: 1 leader_uuid: "5d9e778ae5d448229ec5b655ec13a9a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9e778ae5d448229ec5b655ec13a9a6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 32885 } } }
I20260805 22:46:11.701311 17230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d9e778ae5d448229ec5b655ec13a9a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9e778ae5d448229ec5b655ec13a9a6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 32885 } } }
I20260805 22:46:11.701485 17228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:11.701498 17230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:11.701759 17239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:11.702569 17239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:11.704511 17239 catalog_manager.cc:1347] Generated new cluster ID: d16ec342a0364aec8e8d5dc1bc4fdf31
I20260805 22:46:11.704571 17239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:11.705610 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:11.693955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.23.197.126:32885)
0805 22:46:11.694158 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:11.694165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:11.694190 (+ 25us) server_negotiation.cc:407] Connection header received
0805 22:46:11.694640 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:11.694649 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:11.694704 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:11.694819 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:11.695644 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:11.696703 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:11.697888 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:11.698332 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:11.698414 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:11.701536 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:11.701541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:11.701541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:11.701554 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:11.703869 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:11.704298 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:11.704302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:11.704303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:11.704428 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:11.704700 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:11.704701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:11.704701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:11.705009 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:11.705164 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:11.705291 (+ 127us) server_negotiation.cc:299] Negotiation successful
0805 22:46:11.705435 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260805 22:46:11.712968 17239 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:11.713495 17239 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:11.723871 17239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6: Generated new TSK 0
I20260805 22:46:11.724083 17239 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:46:11.755995 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:32885
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:11.864202 17257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:11.864455 17257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:11.864499 17257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:11.865804 17257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:11.865885 17257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:11.865933 17257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:46:11.867559 17257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:32885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:11.867874 17257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:11.868139 17257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:11.873618 17267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:11.873651 17264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:11.873754 17265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:11.873773 17257 server_base.cc:1034] running on GCE node
I20260805 22:46:11.874169 17257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:11.874436 17257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:11.875607 17257 hybrid_clock.cc:648] HybridClock initialized: now 1785969971875587 us; error 34 us; skew 500 ppm
Aug 05 22:46:11 dist-test-slave-g170 krb5kdc[17090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969971, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:11.878598 17257 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:46:11.879154 17257 webserver.cc:460] Webserver started at http://127.23.197.65:46019/ using document root <none> and password file <none>
I20260805 22:46:11.879420 17257 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:11.879490 17257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:11.879652 17257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:11.880856 17257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "68698cd2892e49adafb451d7c2e91478"
format_stamp: "Formatted at 2026-08-05 22:46:11 on dist-test-slave-g170"
server_key: "91aaa3429d5008dee05ec0fea05f190e"
server_key_iv: "f1501fc257c1eecbc091ef4f16ebb418"
server_key_version: "encryptionkey@0"
I20260805 22:46:11.881237 17257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "68698cd2892e49adafb451d7c2e91478"
format_stamp: "Formatted at 2026-08-05 22:46:11 on dist-test-slave-g170"
server_key: "91aaa3429d5008dee05ec0fea05f190e"
server_key_iv: "f1501fc257c1eecbc091ef4f16ebb418"
server_key_version: "encryptionkey@0"
I20260805 22:46:11.882670 17257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:46:11.883493 17281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:11.883695 17257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:11.883805 17257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "68698cd2892e49adafb451d7c2e91478"
format_stamp: "Formatted at 2026-08-05 22:46:11 on dist-test-slave-g170"
server_key: "91aaa3429d5008dee05ec0fea05f190e"
server_key_iv: "f1501fc257c1eecbc091ef4f16ebb418"
server_key_version: "encryptionkey@0"
I20260805 22:46:11.883881 17257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:11.905684 17257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:11.908214 17257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:11.908393 17257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:11.908663 17257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:11.909035 17257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:11.909096 17257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:11.909132 17257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:11.909161 17257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:11.915738 17257 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:34057
I20260805 22:46:11.915786 17454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:34057 every 8 connection(s)
I20260805 22:46:11.916141 17257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
Aug 05 22:46:11 dist-test-slave-g170 krb5kdc[17090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969971, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:11.922950 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 17257
I20260805 22:46:11.923072 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:46:11.923311 24341 external_mini_cluster.cc:1428] Setting key bb808968b77a22f4ca74ead48a753324
I20260805 22:46:11.926293 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:11.917023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:54749 (local address 127.23.197.126:32885)
0805 22:46:11.917086 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:11.917089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:11.917342 (+ 253us) server_negotiation.cc:407] Connection header received
0805 22:46:11.917816 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:11.917817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:11.917829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:11.917904 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:11.918909 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:11.919474 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:11.920525 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:11.920818 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:11.920885 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:11.923192 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:11.923194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:11.923194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:11.923208 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:11.924693 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:11.925250 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:11.925251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:11.925251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:11.925307 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:11.925603 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:11.925603 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:11.925604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:11.925719 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:11.925774 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:11.926135 (+ 361us) server_negotiation.cc:299] Negotiation successful
0805 22:46:11.926228 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 22:46:11.926575 17458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:11.916962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:32885 (local address 127.23.197.65:54749)
0805 22:46:11.917147 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:11.917194 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:11.917699 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:11.917987 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:11.917991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:11.918607 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:11.918811 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:11.918815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:11.919565 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:11.919566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:11.920426 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:11.920428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:11.920907 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:11.920908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:11.921014 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:11.921543 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:11.921552 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:11.923031 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:11.924818 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:11.924826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:11.924829 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:11.925166 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:11.925381 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:11.925382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:11.925382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:11.925535 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:11.925791 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:11.925798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:11.925974 (+ 176us) client_negotiation.cc:769] Sending connection context
0805 22:46:11.926086 (+ 112us) client_negotiation.cc:240] Negotiation successful
0805 22:46:11.926286 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260805 22:46:11.927224 17455 heartbeater.cc:347] Connected to a master server at 127.23.197.126:32885
I20260805 22:46:11.927332 17455 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:11.927541 17455 heartbeater.cc:511] Master 127.23.197.126:32885 requested a full tablet report, sending...
I20260805 22:46:11.928130 17165 ts_manager.cc:194] Registered new tserver with Master: 68698cd2892e49adafb451d7c2e91478 (127.23.197.65:34057)
I20260805 22:46:11.928969 17165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:54749
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:46:11.952229 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:32885
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:12.037662 17464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:12.037901 17464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:12.037940 17464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:12.039103 17464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:12.039176 17464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:12.039218 17464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:46:12.040680 17464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:32885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:12.040987 17464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:12.041227 17464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:12.046777 17473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:12.046809 17474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:12.047030 17480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:12.047042 17464 server_base.cc:1034] running on GCE node
I20260805 22:46:12.047428 17464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:12.047964 17464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:12.049183 17464 hybrid_clock.cc:648] HybridClock initialized: now 1785969972049171 us; error 34 us; skew 500 ppm
Aug 05 22:46:12 dist-test-slave-g170 krb5kdc[17090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969972, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:12.051756 17464 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:46:12.052287 17464 webserver.cc:460] Webserver started at http://127.23.197.66:33415/ using document root <none> and password file <none>
I20260805 22:46:12.052518 17464 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:12.052588 17464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:12.052745 17464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:12.053859 17464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "3389989ae3a147b59a514abca420a70e"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "c22fa59d41d83711091667b4eb69dc62"
server_key_iv: "862a75410c59dc59ff52d98b127b3904"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.054229 17464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "3389989ae3a147b59a514abca420a70e"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "c22fa59d41d83711091667b4eb69dc62"
server_key_iv: "862a75410c59dc59ff52d98b127b3904"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.055678 17464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:12.056483 17490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.056737 17464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:12.056913 17464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "3389989ae3a147b59a514abca420a70e"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "c22fa59d41d83711091667b4eb69dc62"
server_key_iv: "862a75410c59dc59ff52d98b127b3904"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.057004 17464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:12.071111 17464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:12.073648 17464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:12.073836 17464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:12.074102 17464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:12.074436 17464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:12.074496 17464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.074536 17464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:12.074580 17464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.081353 17464 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:39463
I20260805 22:46:12.081405 17647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:39463 every 8 connection(s)
I20260805 22:46:12.081740 17464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
Aug 05 22:46:12 dist-test-slave-g170 krb5kdc[17090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969972, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:12.088515 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 17464
I20260805 22:46:12.088644 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:46:12.088912 24341 external_mini_cluster.cc:1428] Setting key e8058fb76bf21d3b233c4d9ec143f648
I20260805 22:46:12.092743 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.082551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:42135 (local address 127.23.197.126:32885)
0805 22:46:12.082634 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.082637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.082943 (+ 306us) server_negotiation.cc:407] Connection header received
0805 22:46:12.083469 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.083471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.083482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.083543 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:12.084493 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.085109 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.086102 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.086310 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.086374 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.089239 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:12.089242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:12.089242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:12.089251 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:12.091097 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:12.091533 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:12.091534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:12.091535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:12.091602 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:12.091927 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:12.091928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:12.091929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:12.092083 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:12.092164 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.092560 (+ 396us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.092636 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 22:46:12.093036 17651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.082502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:32885 (local address 127.23.197.66:42135)
0805 22:46:12.082722 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.082761 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:12.083362 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:12.083599 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:12.083602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:12.084196 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:12.084398 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.084402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.085192 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.085193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.086019 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.086021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.086419 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.086420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.086549 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.087321 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:12.087331 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:12.089062 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:12.091186 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:12.091192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:12.091195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:12.091456 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:12.091698 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:12.091699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:12.091699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:12.091813 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:12.092182 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:12.092190 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:12.092408 (+ 218us) client_negotiation.cc:769] Sending connection context
0805 22:46:12.092536 (+ 128us) client_negotiation.cc:240] Negotiation successful
0805 22:46:12.092739 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":126,"threads_started":1}
I20260805 22:46:12.093559 17648 heartbeater.cc:347] Connected to a master server at 127.23.197.126:32885
I20260805 22:46:12.093674 17648 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:12.093883 17648 heartbeater.cc:511] Master 127.23.197.126:32885 requested a full tablet report, sending...
I20260805 22:46:12.094329 17165 ts_manager.cc:194] Registered new tserver with Master: 3389989ae3a147b59a514abca420a70e (127.23.197.66:39463)
I20260805 22:46:12.094897 17165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:42135
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:46:12.119939 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:32885
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:12.205268 17655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:12.205475 17655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:12.205498 17655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:12.206637 17655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:12.206686 17655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:12.206713 17655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:46:12.208137 17655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:32885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:12.208390 17655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:12.208652 17655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:12.213792 17672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:12.213747 17668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:12.213740 17669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:12.213963 17655 server_base.cc:1034] running on GCE node
I20260805 22:46:12.214241 17655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:12.214547 17655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:12.215807 17655 hybrid_clock.cc:648] HybridClock initialized: now 1785969972215793 us; error 56 us; skew 500 ppm
Aug 05 22:46:12 dist-test-slave-g170 krb5kdc[17090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969972, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:12.218906 17655 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:46:12.219393 17655 webserver.cc:460] Webserver started at http://127.23.197.67:38605/ using document root <none> and password file <none>
I20260805 22:46:12.219648 17655 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:12.219700 17655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:12.219846 17655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:12.221078 17655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "2434ed7b20394eb3ac66d87b258fdaa1"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "258079f58ef58aac6e17d1dd6201089d"
server_key_iv: "14938ecc5b847e983751bd458c15053a"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.221470 17655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "2434ed7b20394eb3ac66d87b258fdaa1"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "258079f58ef58aac6e17d1dd6201089d"
server_key_iv: "14938ecc5b847e983751bd458c15053a"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.222972 17655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:46:12.223923 17681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.224159 17655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:12.224267 17655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "2434ed7b20394eb3ac66d87b258fdaa1"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "258079f58ef58aac6e17d1dd6201089d"
server_key_iv: "14938ecc5b847e983751bd458c15053a"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.224328 17655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:12.237816 17655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:12.239877 17655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:12.240051 17655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:12.240316 17655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:12.240660 17655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:12.240723 17655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.240794 17655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:12.240849 17655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.247660 17655 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:33347
I20260805 22:46:12.247740 17856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:33347 every 8 connection(s)
I20260805 22:46:12.248108 17655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
Aug 05 22:46:12 dist-test-slave-g170 krb5kdc[17090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969972, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:12.255961 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 17655
I20260805 22:46:12.256085 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:46:12.256291 24341 external_mini_cluster.cc:1428] Setting key 0faa53dfa4dfa086443dfbf7482b22b7
I20260805 22:46:12.258131 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.249035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:50009 (local address 127.23.197.126:32885)
0805 22:46:12.249133 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.249136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.249331 (+ 195us) server_negotiation.cc:407] Connection header received
0805 22:46:12.249844 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.249846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.249857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.249912 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:12.250953 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.251447 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.252496 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.252752 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.252853 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.254999 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:12.255001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:12.255002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:12.255009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:12.256566 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:12.257095 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:12.257097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:12.257097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:12.257196 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:12.257508 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:12.257509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:12.257509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:12.257655 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:12.257722 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.257992 (+ 270us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.258061 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 22:46:12.258400 17861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.249005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:32885 (local address 127.23.197.67:50009)
0805 22:46:12.249194 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.249238 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:12.249756 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:12.249970 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:12.249973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:12.250631 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:12.250850 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.250855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.251535 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.251536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.252414 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.252416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.252861 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.252862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.252990 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.253480 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:12.253490 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:12.254904 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:12.256649 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:12.256658 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:12.256660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:12.257034 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:12.257272 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:12.257272 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:12.257273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:12.257432 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:12.257723 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:12.257729 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:12.257865 (+ 136us) client_negotiation.cc:769] Sending connection context
0805 22:46:12.257958 (+ 93us) client_negotiation.cc:240] Negotiation successful
0805 22:46:12.258086 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260805 22:46:12.258951 17857 heartbeater.cc:347] Connected to a master server at 127.23.197.126:32885
I20260805 22:46:12.259049 17857 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:12.259204 17857 heartbeater.cc:511] Master 127.23.197.126:32885 requested a full tablet report, sending...
I20260805 22:46:12.259596 17165 ts_manager.cc:194] Registered new tserver with Master: 2434ed7b20394eb3ac66d87b258fdaa1 (127.23.197.67:33347)
I20260805 22:46:12.260063 17165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:50009
I20260805 22:46:12.268721 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:46:12.278693 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.270752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.23.197.126:32885)
0805 22:46:12.270816 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.270818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.270827 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:12.270850 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.270851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.270861 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.270917 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:12.271472 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.272020 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.273126 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.273447 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.273531 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.275970 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:12.275971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:12.275972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:12.275978 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:12.277562 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:12.277935 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:12.277936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:12.277936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:12.277990 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:12.278139 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:12.278139 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:12.278140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:12.278279 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:12.278363 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.278451 (+ 88us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.278549 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260805 22:46:12.280462 17165 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:46:12.282120 17165 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:46:12.288797 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.284519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:33347 (local address 127.0.0.1:39222)
0805 22:46:12.284926 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.284935 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:12.285006 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:12.285321 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:12.285322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:12.285325 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:12.285386 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.285388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.286484 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.286485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.287623 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.287626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.288473 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.288474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.288562 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.288576 (+ 14us) client_negotiation.cc:769] Sending connection context
0805 22:46:12.288642 (+ 66us) client_negotiation.cc:240] Negotiation successful
0805 22:46:12.288692 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":2,"thread_start_us":65,"threads_started":1}
I20260805 22:46:12.289100 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.284909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.23.197.66:39463)
0805 22:46:12.285091 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.285095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.285426 (+ 331us) server_negotiation.cc:407] Connection header received
0805 22:46:12.285549 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.285552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.285567 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.285619 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:12.285865 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.286909 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.288214 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.288705 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.288825 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.288894 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.288979 (+ 85us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.289022 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":80,"threads_started":1}
I20260805 22:46:12.289275 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.284826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39222 (local address 127.23.197.67:33347)
0805 22:46:12.285158 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.285160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.285168 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:12.285201 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.285202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.285214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.285501 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:12.285675 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.286362 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.287714 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.288307 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.288380 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.288439 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.289119 (+ 680us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.289164 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":109,"threads_started":1}
I20260805 22:46:12.289076 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.284529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:39463 (local address 127.0.0.1:52858)
0805 22:46:12.285362 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.285372 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:12.285456 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:12.285736 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:12.285737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:12.285739 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:12.285788 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.285789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.287018 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.287019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.288114 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.288116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.288824 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.288826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.288898 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.288904 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 22:46:12.288963 (+ 59us) client_negotiation.cc:240] Negotiation successful
0805 22:46:12.289006 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":109,"threads_started":1}
I20260805 22:46:12.289547 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.284653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59458 (local address 127.23.197.65:34057)
0805 22:46:12.284878 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.284883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.285094 (+ 211us) server_negotiation.cc:407] Connection header received
0805 22:46:12.285143 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.285147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.285168 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.285257 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:12.285442 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.286641 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.288148 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.288946 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.289010 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.289060 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.289404 (+ 344us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.289454 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":96,"threads_started":1}
I20260805 22:46:12.290230 17553 tablet_service.cc:1467] Processing CreateTablet for tablet 389d7a135a3f4b04b76bdce38c4a2e14 (DEFAULT_TABLE table=test-table [id=4419df2439c64b8189f92087e217c03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:12.290226 17751 tablet_service.cc:1467] Processing CreateTablet for tablet 389d7a135a3f4b04b76bdce38c4a2e14 (DEFAULT_TABLE table=test-table [id=4419df2439c64b8189f92087e217c03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:12.290635 17553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389d7a135a3f4b04b76bdce38c4a2e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:12.290637 17751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389d7a135a3f4b04b76bdce38c4a2e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:12.289332 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.284518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:34057 (local address 127.0.0.1:59458)
0805 22:46:12.284882 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.284935 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:12.285006 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:12.285261 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:12.285264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:12.285266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:12.285341 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.285343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.286763 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.286764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.288052 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.288055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.289097 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.289098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.289161 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.289165 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:12.289216 (+ 51us) client_negotiation.cc:240] Negotiation successful
0805 22:46:12.289255 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":76,"thread_start_us":97,"threads_started":1}
I20260805 22:46:12.293733 17360 tablet_service.cc:1467] Processing CreateTablet for tablet 389d7a135a3f4b04b76bdce38c4a2e14 (DEFAULT_TABLE table=test-table [id=4419df2439c64b8189f92087e217c03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:12.294087 17360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389d7a135a3f4b04b76bdce38c4a2e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:12.294389 17886 tablet_bootstrap.cc:492] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: Bootstrap starting.
I20260805 22:46:12.295023 17886 tablet_bootstrap.cc:654] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:12.295528 17886 log.cc:824] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:12.295912 17887 tablet_bootstrap.cc:492] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: Bootstrap starting.
I20260805 22:46:12.296347 17886 tablet_bootstrap.cc:492] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: No bootstrap required, opened a new log
I20260805 22:46:12.296432 17886 ts_tablet_manager.cc:1397] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:46:12.296681 17887 tablet_bootstrap.cc:654] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:12.297333 17887 log.cc:824] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:12.297341 17889 tablet_bootstrap.cc:492] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478: Bootstrap starting.
I20260805 22:46:12.298043 17889 tablet_bootstrap.cc:654] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:12.298183 17887 tablet_bootstrap.cc:492] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: No bootstrap required, opened a new log
I20260805 22:46:12.298269 17887 ts_tablet_manager.cc:1397] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:46:12.298714 17889 log.cc:824] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:12.298714 17886 raft_consensus.cc:348] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } }
I20260805 22:46:12.298941 17886 raft_consensus.cc:374] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:12.298998 17886 raft_consensus.cc:729] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2434ed7b20394eb3ac66d87b258fdaa1, State: Initialized, Role: FOLLOWER
I20260805 22:46:12.299118 17886 consensus_queue.cc:260] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } }
I20260805 22:46:12.299422 17886 ts_tablet_manager.cc:1428] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:12.299484 17857 heartbeater.cc:503] Master 127.23.197.126:32885 was elected leader, sending a full tablet report...
I20260805 22:46:12.300282 17887 raft_consensus.cc:348] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } }
I20260805 22:46:12.300452 17887 raft_consensus.cc:374] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:12.300501 17887 raft_consensus.cc:729] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3389989ae3a147b59a514abca420a70e, State: Initialized, Role: FOLLOWER
I20260805 22:46:12.300618 17887 consensus_queue.cc:260] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } }
I20260805 22:46:12.301038 17887 ts_tablet_manager.cc:1428] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:12.301046 17648 heartbeater.cc:503] Master 127.23.197.126:32885 was elected leader, sending a full tablet report...
I20260805 22:46:12.301524 17889 tablet_bootstrap.cc:492] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478: No bootstrap required, opened a new log
I20260805 22:46:12.301608 17889 ts_tablet_manager.cc:1397] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 22:46:12.303056 17889 raft_consensus.cc:348] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } }
I20260805 22:46:12.303218 17889 raft_consensus.cc:374] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:12.303268 17889 raft_consensus.cc:729] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68698cd2892e49adafb451d7c2e91478, State: Initialized, Role: FOLLOWER
I20260805 22:46:12.303376 17889 consensus_queue.cc:260] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } }
I20260805 22:46:12.303664 17889 ts_tablet_manager.cc:1428] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:46:12.303714 17455 heartbeater.cc:503] Master 127.23.197.126:32885 was elected leader, sending a full tablet report...
W20260805 22:46:12.332425 17650 tablet.cc:2369] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:46:12.405031 17897 raft_consensus.cc:482] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:46:12.405200 17897 raft_consensus.cc:504] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } }
I20260805 22:46:12.405651 17897 leader_election.cc:288] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2434ed7b20394eb3ac66d87b258fdaa1 (127.23.197.67:33347), 3389989ae3a147b59a514abca420a70e (127.23.197.66:39463)
I20260805 22:46:12.408635 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.405973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:35993 (local address 127.23.197.66:39463)
0805 22:46:12.406029 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.406031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.406093 (+ 62us) server_negotiation.cc:407] Connection header received
0805 22:46:12.406169 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.406172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.406185 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.406245 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:12.406423 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.406893 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.407870 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.408267 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.408357 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.408366 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.408525 (+ 159us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.408561 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260805 22:46:12.408670 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.405926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:58715 (local address 127.23.197.67:33347)
0805 22:46:12.405973 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.405975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.405983 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:12.406013 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.406014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.406024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.406081 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:12.406249 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.406732 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.407794 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.408233 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.408333 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.408345 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.408502 (+ 157us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.408548 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260805 22:46:12.408664 17458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.405807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:33347 (local address 127.23.197.65:58715)
0805 22:46:12.405873 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.405887 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:12.405931 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:12.406085 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:12.406086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:12.406088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:12.406171 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.406173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.406822 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.406823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.407630 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.407632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.408328 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.408329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.408423 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.408426 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:12.408494 (+ 68us) client_negotiation.cc:240] Negotiation successful
0805 22:46:12.408537 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260805 22:46:12.408667 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.405854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:39463 (local address 127.23.197.65:35993)
0805 22:46:12.406003 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.406014 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:12.406101 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:12.406266 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:12.406266 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:12.406269 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:12.406350 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.406352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.406999 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.407000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.407776 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.407778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.408359 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.408360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.408446 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.408450 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:12.408518 (+ 68us) client_negotiation.cc:240] Negotiation successful
0805 22:46:12.408559 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":68,"threads_started":1}
I20260805 22:46:12.409116 17784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389d7a135a3f4b04b76bdce38c4a2e14" candidate_uuid: "68698cd2892e49adafb451d7c2e91478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" is_pre_election: true
I20260805 22:46:12.409312 17784 raft_consensus.cc:2454] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68698cd2892e49adafb451d7c2e91478 in term 0.
I20260805 22:46:12.409461 17585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389d7a135a3f4b04b76bdce38c4a2e14" candidate_uuid: "68698cd2892e49adafb451d7c2e91478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3389989ae3a147b59a514abca420a70e" is_pre_election: true
I20260805 22:46:12.409642 17287 leader_election.cc:302] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2434ed7b20394eb3ac66d87b258fdaa1, 68698cd2892e49adafb451d7c2e91478; no voters:
I20260805 22:46:12.409677 17585 raft_consensus.cc:2454] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68698cd2892e49adafb451d7c2e91478 in term 0.
I20260805 22:46:12.409780 17897 raft_consensus.cc:2781] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:46:12.409869 17897 raft_consensus.cc:482] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:46:12.409899 17897 raft_consensus.cc:3037] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:12.410775 17897 raft_consensus.cc:504] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } }
I20260805 22:46:12.410916 17897 leader_election.cc:288] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [CANDIDATE]: Term 1 election: Requested vote from peers 2434ed7b20394eb3ac66d87b258fdaa1 (127.23.197.67:33347), 3389989ae3a147b59a514abca420a70e (127.23.197.66:39463)
I20260805 22:46:12.411123 17585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389d7a135a3f4b04b76bdce38c4a2e14" candidate_uuid: "68698cd2892e49adafb451d7c2e91478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3389989ae3a147b59a514abca420a70e"
I20260805 22:46:12.411222 17585 raft_consensus.cc:3037] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:12.411962 17585 raft_consensus.cc:2454] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68698cd2892e49adafb451d7c2e91478 in term 1.
I20260805 22:46:12.412099 17784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389d7a135a3f4b04b76bdce38c4a2e14" candidate_uuid: "68698cd2892e49adafb451d7c2e91478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2434ed7b20394eb3ac66d87b258fdaa1"
I20260805 22:46:12.412202 17288 leader_election.cc:302] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3389989ae3a147b59a514abca420a70e, 68698cd2892e49adafb451d7c2e91478; no voters:
I20260805 22:46:12.412194 17784 raft_consensus.cc:3037] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:12.412315 17897 raft_consensus.cc:2781] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:12.412503 17897 raft_consensus.cc:686] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [term 1 LEADER]: Becoming Leader. State: Replica: 68698cd2892e49adafb451d7c2e91478, State: Running, Role: LEADER
I20260805 22:46:12.412601 17897 consensus_queue.cc:237] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } }
I20260805 22:46:12.413193 17784 raft_consensus.cc:2454] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68698cd2892e49adafb451d7c2e91478 in term 1.
I20260805 22:46:12.413522 17165 catalog_manager.cc:5495] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68698cd2892e49adafb451d7c2e91478 (127.23.197.65). New cstate: current_term: 1 leader_uuid: "68698cd2892e49adafb451d7c2e91478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68698cd2892e49adafb451d7c2e91478" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 } health_report { overall_health: UNKNOWN } } }
W20260805 22:46:12.416906 17457 tablet.cc:2369] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:46:12.438349 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.435921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59466 (local address 127.23.197.65:34057)
0805 22:46:12.435988 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.435990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.435999 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:12.436057 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.436057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.436075 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.436137 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:12.436271 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.436798 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.437548 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.437732 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.437783 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.437906 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:12.438062 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:12.438144 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.438183 (+ 39us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.438206 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 22:46:12.440269 17784 raft_consensus.cc:1261] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [term 1 FOLLOWER]: Refusing update from remote peer 68698cd2892e49adafb451d7c2e91478: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:12.440271 17585 raft_consensus.cc:1261] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [term 1 FOLLOWER]: Refusing update from remote peer 68698cd2892e49adafb451d7c2e91478: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:12.440649 17901 consensus_queue.cc:1035] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2434ed7b20394eb3ac66d87b258fdaa1" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 33347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:12.440852 17897 consensus_queue.cc:1035] T 389d7a135a3f4b04b76bdce38c4a2e14 P 68698cd2892e49adafb451d7c2e91478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3389989ae3a147b59a514abca420a70e" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 39463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:12.443488 17905 mvcc.cc:205] Tried to move back new op lower bound from 7315333007112200192 to 7315333007002824704. Current Snapshot: MvccSnapshot[applied={T|T < 7315333007112200192}]
I20260805 22:46:12.444475 17908 mvcc.cc:205] Tried to move back new op lower bound from 7315333007112200192 to 7315333007002824704. Current Snapshot: MvccSnapshot[applied={T|T < 7315333007112200192 or (T in {7315333007112200192})}]
I20260805 22:46:12.444594 17909 mvcc.cc:205] Tried to move back new op lower bound from 7315333007112200192 to 7315333007002824704. Current Snapshot: MvccSnapshot[applied={T|T < 7315333007112200192 or (T in {7315333007112200192})}]
I20260805 22:46:12.445839 17164 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:12.445940 17164 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:12.446745 17164 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d9e778ae5d448229ec5b655ec13a9a6: Sending DeleteTablet for 3 replicas of tablet 389d7a135a3f4b04b76bdce38c4a2e14
I20260805 22:46:12.447098 17360 tablet_service.cc:1514] Processing DeleteTablet for tablet 389d7a135a3f4b04b76bdce38c4a2e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:12 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:59458
I20260805 22:46:12.447120 17751 tablet_service.cc:1514] Processing DeleteTablet for tablet 389d7a135a3f4b04b76bdce38c4a2e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:12 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:39222
I20260805 22:46:12.447383 17553 tablet_service.cc:1514] Processing DeleteTablet for tablet 389d7a135a3f4b04b76bdce38c4a2e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:12 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:52858
I20260805 22:46:12.447438 17921 tablet_replica.cc:330] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: stopping tablet replica
I20260805 22:46:12.447535 17921 raft_consensus.cc:2229] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:46:12.447625 17921 raft_consensus.cc:2258] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:12.447638 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 17257
I20260805 22:46:12.452255 17921 ts_tablet_manager.cc:1905] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:12.453670 17923 tablet_replica.cc:330] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: stopping tablet replica
I20260805 22:46:12.453795 17923 raft_consensus.cc:2229] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:46:12.453945 17923 raft_consensus.cc:2258] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:12.453975 17921 ts_tablet_manager.cc:1918] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:12.454150 17921 log.cc:1196] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/389d7a135a3f4b04b76bdce38c4a2e14
I20260805 22:46:12.454592 17921 ts_tablet_manager.cc:1939] T 389d7a135a3f4b04b76bdce38c4a2e14 P 2434ed7b20394eb3ac66d87b258fdaa1: Deleting consensus metadata
I20260805 22:46:12.455150 17133 catalog_manager.cc:4841] TS 2434ed7b20394eb3ac66d87b258fdaa1 (127.23.197.67:33347): tablet 389d7a135a3f4b04b76bdce38c4a2e14 (table test-table [id=4419df2439c64b8189f92087e217c03a]) successfully deleted
I20260805 22:46:12.456108 17923 ts_tablet_manager.cc:1905] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:12.457868 17923 ts_tablet_manager.cc:1918] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:12.457931 17923 log.cc:1196] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/389d7a135a3f4b04b76bdce38c4a2e14
I20260805 22:46:12.458197 17923 ts_tablet_manager.cc:1939] T 389d7a135a3f4b04b76bdce38c4a2e14 P 3389989ae3a147b59a514abca420a70e: Deleting consensus metadata
I20260805 22:46:12.458643 17134 catalog_manager.cc:4841] TS 3389989ae3a147b59a514abca420a70e (127.23.197.66:39463): tablet 389d7a135a3f4b04b76bdce38c4a2e14 (table test-table [id=4419df2439c64b8189f92087e217c03a]) successfully deleted
W20260805 22:46:12.461946 17132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:34057 (error 108)
I20260805 22:46:12.462179 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 17464
I20260805 22:46:12.467684 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.467482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:34057 (local address 127.0.0.1:59472)
0805 22:46:12.467586 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.467647 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:34057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260805 22:46:12.467908 17132 catalog_manager.cc:4584] TS 68698cd2892e49adafb451d7c2e91478 (127.23.197.65:34057): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 389d7a135a3f4b04b76bdce38c4a2e14: Network error: Client connection negotiation failed: client connection to 127.23.197.65:34057: connect: Connection refused (error 111)
I20260805 22:46:12.471276 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 17655
I20260805 22:46:12.482227 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 17106
2026-08-05T22:46:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2307 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-05T22:46:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:12Z Disabled control of system clock
I20260805 22:46:12.514797 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36001
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:34787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:36001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 22:46:12.597172 17929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:12.597396 17929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:12.597417 17929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:12.598527 17929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:12.598563 17929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:12.598577 17929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:12.598588 17929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:12.598601 17929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:12.599996 17929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:34787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:36001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:12.600211 17929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:12.600448 17929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:12.605185 17939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:12.605221 17944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:12.605247 17940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:12.605752 17929 server_base.cc:1034] running on GCE node
I20260805 22:46:12.605994 17929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:12.606251 17929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:12.607390 17929 hybrid_clock.cc:648] HybridClock initialized: now 1785969972607366 us; error 48 us; skew 500 ppm
I20260805 22:46:12.608662 17929 webserver.cc:460] Webserver started at http://127.23.197.126:46005/ using document root <none> and password file <none>
I20260805 22:46:12.608960 17929 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:12.609035 17929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:12.609197 17929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:12.610320 17929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "bbf7095468e149b58e6e1cd40ecab187"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "0e0a8aa6bedb565eb843248699ed2200"
server_key_iv: "096f674c7ea8455cb19986e8a4752f9e"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.610756 17929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "bbf7095468e149b58e6e1cd40ecab187"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "0e0a8aa6bedb565eb843248699ed2200"
server_key_iv: "096f674c7ea8455cb19986e8a4752f9e"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.612104 17929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:12.612890 17952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.613068 17929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:12.613170 17929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "bbf7095468e149b58e6e1cd40ecab187"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "0e0a8aa6bedb565eb843248699ed2200"
server_key_iv: "096f674c7ea8455cb19986e8a4752f9e"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.613256 17929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:12.642819 17929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:12.643182 17929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:12.643345 17929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:12.647827 17929 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:36001
I20260805 22:46:12.647872 18023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:36001 every 8 connection(s)
I20260805 22:46:12.648267 17929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:12.649377 18024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:12.650039 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 17929
I20260805 22:46:12.650195 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:12.650410 24341 external_mini_cluster.cc:1428] Setting key 2420a08c94f17c7492690eacb3c7082a
I20260805 22:46:12.652958 18024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187: Bootstrap starting.
I20260805 22:46:12.653666 18024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:12.653991 18024 log.cc:824] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:12.654749 18024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187: No bootstrap required, opened a new log
I20260805 22:46:12.656874 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.651100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.23.197.126:36001)
0805 22:46:12.651349 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.651358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.651399 (+ 41us) server_negotiation.cc:407] Connection header received
0805 22:46:12.651981 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.651991 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.652077 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.652232 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:12.652475 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.653790 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.654630 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.655214 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.655357 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.656083 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:12.656088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:12.656088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:12.656306 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:12.656349 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:12.656364 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:12.656436 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.656540 (+ 104us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.656623 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260805 22:46:12.657313 18024 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf7095468e149b58e6e1cd40ecab187" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36001 } }
I20260805 22:46:12.657464 18024 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:12.657510 18024 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbf7095468e149b58e6e1cd40ecab187, State: Initialized, Role: FOLLOWER
I20260805 22:46:12.657634 18024 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf7095468e149b58e6e1cd40ecab187" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36001 } }
I20260805 22:46:12.657716 18024 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:12.657758 18024 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:12.657811 18024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:12.658336 18024 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf7095468e149b58e6e1cd40ecab187" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36001 } }
I20260805 22:46:12.658423 18024 leader_election.cc:302] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbf7095468e149b58e6e1cd40ecab187; no voters:
I20260805 22:46:12.658601 18024 leader_election.cc:288] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:12.658660 18029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:12.658885 18024 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:12.658910 18029 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [term 1 LEADER]: Becoming Leader. State: Replica: bbf7095468e149b58e6e1cd40ecab187, State: Running, Role: LEADER
I20260805 22:46:12.659139 18029 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf7095468e149b58e6e1cd40ecab187" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36001 } }
I20260805 22:46:12.659548 18032 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbf7095468e149b58e6e1cd40ecab187. Latest consensus state: current_term: 1 leader_uuid: "bbf7095468e149b58e6e1cd40ecab187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf7095468e149b58e6e1cd40ecab187" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36001 } } }
I20260805 22:46:12.659545 18030 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbf7095468e149b58e6e1cd40ecab187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf7095468e149b58e6e1cd40ecab187" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36001 } } }
I20260805 22:46:12.659693 18032 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:12.659709 18030 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:12.660030 18043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:12.660578 18043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:12.662266 18043 catalog_manager.cc:1347] Generated new cluster ID: 5e735f41b51146019651770c17280547
I20260805 22:46:12.662324 18043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:12.671736 18043 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:12.672284 18043 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:12.678185 18043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187: Generated new TSK 0
I20260805 22:46:12.678366 18043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:12.680457 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36001
--builtin_ntp_servers=127.23.197.84:34787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 22:46:12.767896 18055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:12.768112 18055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:12.768143 18055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:12.769398 18055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:12.769441 18055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:12.769469 18055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:46:12.770874 18055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:34787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:12.771176 18055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:12.771512 18055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:12.777679 18060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:12.777674 18062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:12.777892 18055 server_base.cc:1034] running on GCE node
W20260805 22:46:12.777971 18067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:12.778208 18055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:12.778525 18055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:12.779701 18055 hybrid_clock.cc:648] HybridClock initialized: now 1785969972779678 us; error 39 us; skew 500 ppm
I20260805 22:46:12.781213 18055 webserver.cc:460] Webserver started at http://127.23.197.65:46849/ using document root <none> and password file <none>
I20260805 22:46:12.781467 18055 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:12.781538 18055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:12.781694 18055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:12.782868 18055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "350db174883741ecaad42777888ed9ba"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "5a26ee970d4c83b0bf2dfde772e09daa"
server_key_iv: "8b1ace6d7880bf7e539ceb1c09df0e6d"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.783248 18055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "350db174883741ecaad42777888ed9ba"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "5a26ee970d4c83b0bf2dfde772e09daa"
server_key_iv: "8b1ace6d7880bf7e539ceb1c09df0e6d"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.784600 18055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:12.785501 18074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.785704 18055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:12.785821 18055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "350db174883741ecaad42777888ed9ba"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "5a26ee970d4c83b0bf2dfde772e09daa"
server_key_iv: "8b1ace6d7880bf7e539ceb1c09df0e6d"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.785897 18055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:12.802980 18055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:12.803373 18055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:12.803526 18055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:12.803808 18055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:12.804203 18055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:12.804248 18055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.804296 18055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:12.804322 18055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.810859 18055 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:34177
I20260805 22:46:12.810902 18247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:34177 every 8 connection(s)
I20260805 22:46:12.811257 18055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:46:12.817328 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 18055
I20260805 22:46:12.817476 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:46:12.817682 24341 external_mini_cluster.cc:1428] Setting key 700cc4bd2766a99a9507d7cd58cab780
I20260805 22:46:12.817845 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.812072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:53987 (local address 127.23.197.126:36001)
0805 22:46:12.812122 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.812124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.812418 (+ 294us) server_negotiation.cc:407] Connection header received
0805 22:46:12.813036 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.813037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.813061 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.813115 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:12.814225 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.814817 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.816058 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.816312 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.816408 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.817530 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:12.817532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:12.817532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:12.817538 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:12.817556 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:12.817560 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:12.817615 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.817715 (+ 100us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.817756 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:46:12.818099 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.812071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36001 (local address 127.23.197.65:53987)
0805 22:46:12.812284 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.812320 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:12.812909 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:12.813191 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:12.813194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:12.813202 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:12.814091 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.814099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.814931 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.814933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.815959 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.815962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.816434 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.816435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.816580 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.817315 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:46:12.817316 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:12.817346 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:46:12.817383 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:12.817622 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:12.817628 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:12.817631 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:12.817717 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 22:46:12.817842 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260805 22:46:12.818711 18250 heartbeater.cc:347] Connected to a master server at 127.23.197.126:36001
I20260805 22:46:12.818843 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36001
--builtin_ntp_servers=127.23.197.84:34787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 22:46:12.818864 18250 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:12.819077 18250 heartbeater.cc:511] Master 127.23.197.126:36001 requested a full tablet report, sending...
I20260805 22:46:12.819638 17979 ts_manager.cc:194] Registered new tserver with Master: 350db174883741ecaad42777888ed9ba (127.23.197.65:34177)
I20260805 22:46:12.820395 17979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.65:53987
W20260805 22:46:12.906430 18258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:12.906622 18258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:12.906643 18258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:12.907819 18258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:12.907860 18258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:12.907886 18258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:46:12.909272 18258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:34787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:12.909448 18258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:12.909699 18258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:12.914765 18266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:12.914809 18268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:12.914907 18258 server_base.cc:1034] running on GCE node
W20260805 22:46:12.914968 18265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:12.915181 18258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:12.915431 18258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:12.916589 18258 hybrid_clock.cc:648] HybridClock initialized: now 1785969972916581 us; error 34 us; skew 500 ppm
I20260805 22:46:12.917940 18258 webserver.cc:460] Webserver started at http://127.23.197.66:36989/ using document root <none> and password file <none>
I20260805 22:46:12.918206 18258 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:12.918258 18258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:12.918404 18258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:12.919445 18258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "e49032ee96ca42f1982a29f221bd5c02"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "3419c53e9211dcee69249c28d5ef419f"
server_key_iv: "454f0e7128cc2b5692b54e3cbc8ab14b"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.919813 18258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "e49032ee96ca42f1982a29f221bd5c02"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "3419c53e9211dcee69249c28d5ef419f"
server_key_iv: "454f0e7128cc2b5692b54e3cbc8ab14b"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.921201 18258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:12.921984 18277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.922151 18258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260805 22:46:12.922246 18258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "e49032ee96ca42f1982a29f221bd5c02"
format_stamp: "Formatted at 2026-08-05 22:46:12 on dist-test-slave-g170"
server_key: "3419c53e9211dcee69249c28d5ef419f"
server_key_iv: "454f0e7128cc2b5692b54e3cbc8ab14b"
server_key_version: "encryptionkey@0"
I20260805 22:46:12.922329 18258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:12.936424 18258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:12.936748 18258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:12.937022 18258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:12.937273 18258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:12.937630 18258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:12.937690 18258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.937732 18258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:12.937760 18258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:12.943513 18258 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:42983
I20260805 22:46:12.943564 18454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:42983 every 8 connection(s)
I20260805 22:46:12.943919 18258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:46:12.945185 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 18258
I20260805 22:46:12.945276 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:46:12.945508 24341 external_mini_cluster.cc:1428] Setting key 1e33ef14b83bf6c4430eb602ffc56bb5
I20260805 22:46:12.946453 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36001
--builtin_ntp_servers=127.23.197.84:34787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 22:46:12.949581 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.944695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:58121 (local address 127.23.197.126:36001)
0805 22:46:12.944776 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:12.944779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:12.945006 (+ 227us) server_negotiation.cc:407] Connection header received
0805 22:46:12.945646 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:12.945647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:12.945662 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:12.945729 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:12.946473 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.947070 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.948075 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.948290 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.948367 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.949314 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:12.949315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:12.949316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:12.949320 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:12.949334 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:12.949337 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:12.949416 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:12.949498 (+ 82us) server_negotiation.cc:299] Negotiation successful
0805 22:46:12.949524 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260805 22:46:12.949864 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:12.944655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36001 (local address 127.23.197.66:58121)
0805 22:46:12.944854 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:12.944896 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:12.945523 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:12.945800 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:12.945804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:12.945814 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:12.946371 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.946376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.947173 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.947175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.947978 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:12.947979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:12.948389 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:12.948391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:12.948532 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:12.949139 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:46:12.949140 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:12.949168 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:46:12.949206 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:12.949420 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:12.949426 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:12.949429 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:12.949501 (+ 72us) client_negotiation.cc:240] Negotiation successful
0805 22:46:12.949612 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260805 22:46:12.950425 18455 heartbeater.cc:347] Connected to a master server at 127.23.197.126:36001
I20260805 22:46:12.950519 18455 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:12.950752 18455 heartbeater.cc:511] Master 127.23.197.126:36001 requested a full tablet report, sending...
I20260805 22:46:12.951145 17979 ts_manager.cc:194] Registered new tserver with Master: e49032ee96ca42f1982a29f221bd5c02 (127.23.197.66:42983)
I20260805 22:46:12.951488 17979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.66:58121
W20260805 22:46:13.033026 18459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:13.033257 18459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:13.033293 18459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:13.034406 18459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:13.034472 18459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:13.034512 18459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:46:13.035974 18459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:34787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:13.036257 18459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:13.036566 18459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:13.041656 18467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:13.041680 18466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:13.041709 18469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:13.041770 18459 server_base.cc:1034] running on GCE node
I20260805 22:46:13.042151 18459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:13.042356 18459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:13.043538 18459 hybrid_clock.cc:648] HybridClock initialized: now 1785969973043529 us; error 29 us; skew 500 ppm
I20260805 22:46:13.044960 18459 webserver.cc:460] Webserver started at http://127.23.197.67:43333/ using document root <none> and password file <none>
I20260805 22:46:13.045205 18459 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:13.045254 18459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:13.045401 18459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:13.046465 18459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "8ce78ba3120b432c8b32e1bd218a9ea7"
format_stamp: "Formatted at 2026-08-05 22:46:13 on dist-test-slave-g170"
server_key: "6ab8e96bd662c100a4e500871997a470"
server_key_iv: "2dcfea66b2f419129688bec9ffd33247"
server_key_version: "encryptionkey@0"
I20260805 22:46:13.046830 18459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "8ce78ba3120b432c8b32e1bd218a9ea7"
format_stamp: "Formatted at 2026-08-05 22:46:13 on dist-test-slave-g170"
server_key: "6ab8e96bd662c100a4e500871997a470"
server_key_iv: "2dcfea66b2f419129688bec9ffd33247"
server_key_version: "encryptionkey@0"
I20260805 22:46:13.048117 18459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:13.048894 18478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:13.049072 18459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:13.049175 18459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "8ce78ba3120b432c8b32e1bd218a9ea7"
format_stamp: "Formatted at 2026-08-05 22:46:13 on dist-test-slave-g170"
server_key: "6ab8e96bd662c100a4e500871997a470"
server_key_iv: "2dcfea66b2f419129688bec9ffd33247"
server_key_version: "encryptionkey@0"
I20260805 22:46:13.049259 18459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:13.061517 18459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:13.061805 18459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:13.061950 18459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:13.062170 18459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:13.062500 18459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:13.062554 18459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:13.062593 18459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:13.062621 18459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:13.068755 18459 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:44895
I20260805 22:46:13.068825 18647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:44895 every 8 connection(s)
I20260805 22:46:13.069137 18459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:46:13.073031 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 18459
I20260805 22:46:13.073156 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:46:13.073355 24341 external_mini_cluster.cc:1428] Setting key 4092c341fc48eb2a8ecf2aad33bd8e5a
I20260805 22:46:13.074877 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.069878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:57587 (local address 127.23.197.126:36001)
0805 22:46:13.069931 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:13.069932 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:13.070176 (+ 244us) server_negotiation.cc:407] Connection header received
0805 22:46:13.070723 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:13.070724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:13.070737 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:13.070796 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:13.071490 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.072021 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.073272 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.073512 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.073610 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.074537 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:13.074539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:13.074539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:13.074543 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:13.074559 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:13.074563 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:13.074612 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:13.074759 (+ 147us) server_negotiation.cc:299] Negotiation successful
0805 22:46:13.074794 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:46:13.075099 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.069861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36001 (local address 127.23.197.67:57587)
0805 22:46:13.070050 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:13.070084 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:13.070603 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:13.070862 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:13.070866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:13.070877 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:13.071386 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.071391 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.072127 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.072128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.073106 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.073110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.073633 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.073634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.073828 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.074380 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:46:13.074380 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:13.074408 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:46:13.074448 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:13.074620 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:13.074625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:13.074628 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:13.074760 (+ 132us) client_negotiation.cc:240] Negotiation successful
0805 22:46:13.074864 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260805 22:46:13.075585 18648 heartbeater.cc:347] Connected to a master server at 127.23.197.126:36001
I20260805 22:46:13.075680 18648 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:13.075821 18648 heartbeater.cc:511] Master 127.23.197.126:36001 requested a full tablet report, sending...
I20260805 22:46:13.076215 17979 ts_manager.cc:194] Registered new tserver with Master: 8ce78ba3120b432c8b32e1bd218a9ea7 (127.23.197.67:44895)
I20260805 22:46:13.076670 17979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.67:57587
I20260805 22:46:13.085542 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:46:13.090329 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.087015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40560 (local address 127.23.197.126:36001)
0805 22:46:13.087093 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:13.087095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:13.087225 (+ 130us) server_negotiation.cc:407] Connection header received
0805 22:46:13.087254 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:13.087254 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:13.087266 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:13.087335 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:13.087501 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.088083 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.088980 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.089277 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.089344 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.090134 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:13.090135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:13.090135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:13.090140 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:13.090154 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:13.090157 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:13.090207 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:13.090252 (+ 45us) server_negotiation.cc:299] Negotiation successful
0805 22:46:13.090273 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 22:46:13.091993 17979 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:46:13.093612 17979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:46:13.099939 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.095973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38392 (local address 127.23.197.66:42983)
0805 22:46:13.096170 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:13.096174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:13.096182 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:13.096213 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:13.096214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:13.096227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:13.096314 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:13.096493 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.097352 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.098644 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.099386 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.099462 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.099553 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:13.099807 (+ 254us) server_negotiation.cc:299] Negotiation successful
0805 22:46:13.099850 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
I20260805 22:46:13.099937 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.095755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:42983 (local address 127.0.0.1:38392)
0805 22:46:13.096004 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:13.096012 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:13.096050 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:13.096333 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:13.096336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:13.096338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:13.096400 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.096402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.097445 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.097446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.098523 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.098525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.099500 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.099501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.099665 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.099681 (+ 16us) client_negotiation.cc:769] Sending connection context
0805 22:46:13.099792 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 22:46:13.099853 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":78,"threads_started":1}
I20260805 22:46:13.100355 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.095683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:44895 (local address 127.0.0.1:53188)
0805 22:46:13.095791 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:13.095832 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:13.095906 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:13.096597 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:13.096598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:13.096600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:13.096641 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.096642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.098015 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.098016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.099061 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.099064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.100094 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.100095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.100191 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.100200 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 22:46:13.100230 (+ 30us) client_negotiation.cc:240] Negotiation successful
0805 22:46:13.100284 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":63,"threads_started":1}
I20260805 22:46:13.100749 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.095988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.23.197.67:44895)
0805 22:46:13.096461 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:13.096465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:13.096474 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:13.096509 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:13.096511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:13.096525 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:13.096600 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:13.096710 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.097913 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.099167 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.099969 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.100479 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.100583 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:13.100611 (+ 28us) server_negotiation.cc:299] Negotiation successful
0805 22:46:13.100654 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":80,"threads_started":1}
I20260805 22:46:13.100977 18359 tablet_service.cc:1467] Processing CreateTablet for tablet df3a27db8a294040998cbc713bbd3178 (DEFAULT_TABLE table=test-table [id=eb71233b817049a58db17d59386fcd84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:13.101413 18359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3a27db8a294040998cbc713bbd3178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:13.101459 18555 tablet_service.cc:1467] Processing CreateTablet for tablet df3a27db8a294040998cbc713bbd3178 (DEFAULT_TABLE table=test-table [id=eb71233b817049a58db17d59386fcd84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:13.101595 18669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.095714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:34177 (local address 127.0.0.1:47504)
0805 22:46:13.095877 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:13.095887 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:13.095927 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:13.096661 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:13.096662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:13.096664 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:13.096710 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.096711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.099172 (+ 2461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.099173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.100218 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.100221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.101349 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.101350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.101414 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.101418 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:13.101481 (+ 63us) client_negotiation.cc:240] Negotiation successful
0805 22:46:13.101523 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260805 22:46:13.101641 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.096184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47504 (local address 127.23.197.65:34177)
0805 22:46:13.096347 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:13.096352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:13.096364 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:46:13.096406 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:13.096409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:13.096424 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:13.096510 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:13.096868 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.097931 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.100322 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.101245 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.101337 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.101430 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:13.101496 (+ 66us) server_negotiation.cc:299] Negotiation successful
0805 22:46:13.101542 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260805 22:46:13.101850 18555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3a27db8a294040998cbc713bbd3178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:13.102336 18150 tablet_service.cc:1467] Processing CreateTablet for tablet df3a27db8a294040998cbc713bbd3178 (DEFAULT_TABLE table=test-table [id=eb71233b817049a58db17d59386fcd84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:13.102699 18150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3a27db8a294040998cbc713bbd3178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:13.104584 18675 tablet_bootstrap.cc:492] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: Bootstrap starting.
I20260805 22:46:13.105386 18675 tablet_bootstrap.cc:654] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:13.105391 18678 tablet_bootstrap.cc:492] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: Bootstrap starting.
I20260805 22:46:13.105901 18675 log.cc:824] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:13.106115 18678 tablet_bootstrap.cc:654] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:13.106196 18680 tablet_bootstrap.cc:492] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba: Bootstrap starting.
I20260805 22:46:13.106575 18678 log.cc:824] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:13.106979 18680 tablet_bootstrap.cc:654] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:13.107131 18675 tablet_bootstrap.cc:492] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: No bootstrap required, opened a new log
I20260805 22:46:13.107209 18675 ts_tablet_manager.cc:1397] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 22:46:13.107319 18678 tablet_bootstrap.cc:492] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: No bootstrap required, opened a new log
I20260805 22:46:13.107368 18678 ts_tablet_manager.cc:1397] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:46:13.107386 18680 log.cc:824] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:13.108330 18680 tablet_bootstrap.cc:492] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba: No bootstrap required, opened a new log
I20260805 22:46:13.108410 18680 ts_tablet_manager.cc:1397] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:46:13.109727 18678 raft_consensus.cc:348] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } }
I20260805 22:46:13.109902 18678 raft_consensus.cc:374] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:13.109973 18678 raft_consensus.cc:729] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e49032ee96ca42f1982a29f221bd5c02, State: Initialized, Role: FOLLOWER
I20260805 22:46:13.110100 18675 raft_consensus.cc:348] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } }
I20260805 22:46:13.110169 18678 consensus_queue.cc:260] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } }
I20260805 22:46:13.110304 18675 raft_consensus.cc:374] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:13.110368 18675 raft_consensus.cc:729] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce78ba3120b432c8b32e1bd218a9ea7, State: Initialized, Role: FOLLOWER
I20260805 22:46:13.110512 18675 consensus_queue.cc:260] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } }
I20260805 22:46:13.110592 18678 ts_tablet_manager.cc:1428] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:13.110709 18455 heartbeater.cc:503] Master 127.23.197.126:36001 was elected leader, sending a full tablet report...
I20260805 22:46:13.110884 18675 ts_tablet_manager.cc:1428] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260805 22:46:13.111008 18648 heartbeater.cc:503] Master 127.23.197.126:36001 was elected leader, sending a full tablet report...
I20260805 22:46:13.111511 18680 raft_consensus.cc:348] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } }
I20260805 22:46:13.111662 18680 raft_consensus.cc:374] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:13.111749 18680 raft_consensus.cc:729] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350db174883741ecaad42777888ed9ba, State: Initialized, Role: FOLLOWER
I20260805 22:46:13.111920 18680 consensus_queue.cc:260] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } }
I20260805 22:46:13.112231 18680 ts_tablet_manager.cc:1428] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260805 22:46:13.112387 18250 heartbeater.cc:503] Master 127.23.197.126:36001 was elected leader, sending a full tablet report...
W20260805 22:46:13.194528 18457 tablet.cc:2369] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:46:13.216231 18690 raft_consensus.cc:482] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:46:13.216391 18690 raft_consensus.cc:504] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } }
I20260805 22:46:13.216871 18690 leader_election.cc:288] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 350db174883741ecaad42777888ed9ba (127.23.197.65:34177), e49032ee96ca42f1982a29f221bd5c02 (127.23.197.66:42983)
I20260805 22:46:13.219724 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.217062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:34177 (local address 127.23.197.67:36797)
0805 22:46:13.217124 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:13.217136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:13.217181 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:13.217393 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:13.217394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:13.217396 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:13.217460 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.217462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.218151 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.218153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.218910 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.218912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.219477 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.219478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.219548 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.219551 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:13.219610 (+ 59us) client_negotiation.cc:240] Negotiation successful
0805 22:46:13.219654 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260805 22:46:13.219735 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.217206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:36797 (local address 127.23.197.65:34177)
0805 22:46:13.217261 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:13.217263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:13.217271 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:13.217301 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:13.217302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:13.217315 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:13.217374 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:13.217557 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.218048 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.219009 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.219389 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.219478 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.219487 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:13.219637 (+ 150us) server_negotiation.cc:299] Negotiation successful
0805 22:46:13.219677 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260805 22:46:13.219986 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.217072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:42983 (local address 127.23.197.67:41999)
0805 22:46:13.217424 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:13.217433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:13.217485 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:13.217612 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:13.217612 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:13.217613 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:13.217718 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.217720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.218365 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.218366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.219106 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:13.219108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.219670 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.219672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:13.219763 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.219767 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:13.219839 (+ 72us) client_negotiation.cc:240] Negotiation successful
0805 22:46:13.219889 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":160,"thread_start_us":115,"threads_started":1}
I20260805 22:46:13.219993 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.217162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:41999 (local address 127.23.197.66:42983)
0805 22:46:13.217237 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:13.217239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:13.217485 (+ 246us) server_negotiation.cc:407] Connection header received
0805 22:46:13.217541 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:13.217542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:13.217552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:13.217608 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:13.217799 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.218259 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.219213 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.219581 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.219671 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.219679 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:13.219884 (+ 205us) server_negotiation.cc:299] Negotiation successful
0805 22:46:13.219932 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 22:46:13.220086 18182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3a27db8a294040998cbc713bbd3178" candidate_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350db174883741ecaad42777888ed9ba" is_pre_election: true
I20260805 22:46:13.220292 18182 raft_consensus.cc:2454] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ce78ba3120b432c8b32e1bd218a9ea7 in term 0.
I20260805 22:46:13.220480 18381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3a27db8a294040998cbc713bbd3178" candidate_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e49032ee96ca42f1982a29f221bd5c02" is_pre_election: true
I20260805 22:46:13.220563 18482 leader_election.cc:302] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350db174883741ecaad42777888ed9ba, 8ce78ba3120b432c8b32e1bd218a9ea7; no voters:
I20260805 22:46:13.220659 18381 raft_consensus.cc:2454] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ce78ba3120b432c8b32e1bd218a9ea7 in term 0.
I20260805 22:46:13.220687 18690 raft_consensus.cc:2781] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:46:13.220754 18690 raft_consensus.cc:482] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:46:13.220907 18690 raft_consensus.cc:3037] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:13.221674 18690 raft_consensus.cc:504] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } }
I20260805 22:46:13.221822 18690 leader_election.cc:288] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [CANDIDATE]: Term 1 election: Requested vote from peers 350db174883741ecaad42777888ed9ba (127.23.197.65:34177), e49032ee96ca42f1982a29f221bd5c02 (127.23.197.66:42983)
I20260805 22:46:13.222003 18381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3a27db8a294040998cbc713bbd3178" candidate_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e49032ee96ca42f1982a29f221bd5c02"
I20260805 22:46:13.222023 18182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3a27db8a294040998cbc713bbd3178" candidate_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350db174883741ecaad42777888ed9ba"
I20260805 22:46:13.222112 18381 raft_consensus.cc:3037] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:13.222110 18182 raft_consensus.cc:3037] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:13.222880 18182 raft_consensus.cc:2454] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ce78ba3120b432c8b32e1bd218a9ea7 in term 1.
I20260805 22:46:13.222882 18381 raft_consensus.cc:2454] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ce78ba3120b432c8b32e1bd218a9ea7 in term 1.
I20260805 22:46:13.223050 18482 leader_election.cc:302] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350db174883741ecaad42777888ed9ba, 8ce78ba3120b432c8b32e1bd218a9ea7; no voters:
I20260805 22:46:13.223138 18690 raft_consensus.cc:2781] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:13.223289 18690 raft_consensus.cc:686] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 1 LEADER]: Becoming Leader. State: Replica: 8ce78ba3120b432c8b32e1bd218a9ea7, State: Running, Role: LEADER
I20260805 22:46:13.223398 18690 consensus_queue.cc:237] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } }
I20260805 22:46:13.224252 17978 catalog_manager.cc:5495] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ce78ba3120b432c8b32e1bd218a9ea7 (127.23.197.67). New cstate: current_term: 1 leader_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce78ba3120b432c8b32e1bd218a9ea7" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 } health_report { overall_health: UNKNOWN } } }
I20260805 22:46:13.247385 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.245221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.23.197.67:44895)
0805 22:46:13.245262 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:13.245263 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:13.245330 (+ 67us) server_negotiation.cc:407] Connection header received
0805 22:46:13.245387 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:13.245388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:13.245404 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:13.245467 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:13.245585 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.246036 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.246743 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:13.246918 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:13.246967 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:13.247098 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:13.247197 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:13.247257 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:13.247292 (+ 35us) server_negotiation.cc:299] Negotiation successful
0805 22:46:13.247306 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260805 22:46:13.249241 18381 raft_consensus.cc:1261] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [term 1 FOLLOWER]: Refusing update from remote peer 8ce78ba3120b432c8b32e1bd218a9ea7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:13.249257 18182 raft_consensus.cc:1261] T df3a27db8a294040998cbc713bbd3178 P 350db174883741ecaad42777888ed9ba [term 1 FOLLOWER]: Refusing update from remote peer 8ce78ba3120b432c8b32e1bd218a9ea7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:13.249586 18694 consensus_queue.cc:1035] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e49032ee96ca42f1982a29f221bd5c02" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 42983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:13.249727 18690 consensus_queue.cc:1035] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "350db174883741ecaad42777888ed9ba" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:13.252039 18698 mvcc.cc:205] Tried to move back new op lower bound from 7315333010425999360 to 7315333010323824640. Current Snapshot: MvccSnapshot[applied={T|T < 7315333010425999360}]
I20260805 22:46:13.253259 18700 mvcc.cc:205] Tried to move back new op lower bound from 7315333010425999360 to 7315333010323824640. Current Snapshot: MvccSnapshot[applied={T|T < 7315333010425999360 or (T in {7315333010425999360})}]
I20260805 22:46:13.253507 18701 mvcc.cc:205] Tried to move back new op lower bound from 7315333010425999360 to 7315333010323824640. Current Snapshot: MvccSnapshot[applied={T|T < 7315333010425999360 or (T in {7315333010425999360})}]
I20260805 22:46:13.254583 17978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:13.254725 17978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:13.255635 17978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbf7095468e149b58e6e1cd40ecab187: Sending DeleteTablet for 3 replicas of tablet df3a27db8a294040998cbc713bbd3178
I20260805 22:46:13.256078 18555 tablet_service.cc:1514] Processing DeleteTablet for tablet df3a27db8a294040998cbc713bbd3178 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:13 UTC) from {username='slave'} at 127.0.0.1:53188
I20260805 22:46:13.256100 18359 tablet_service.cc:1514] Processing DeleteTablet for tablet df3a27db8a294040998cbc713bbd3178 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:13 UTC) from {username='slave'} at 127.0.0.1:38392
I20260805 22:46:13.256369 18150 tablet_service.cc:1514] Processing DeleteTablet for tablet df3a27db8a294040998cbc713bbd3178 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:13 UTC) from {username='slave'} at 127.0.0.1:47504
I20260805 22:46:13.256507 18712 tablet_replica.cc:330] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: stopping tablet replica
I20260805 22:46:13.256616 18712 raft_consensus.cc:2229] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:46:13.256618 18714 tablet_replica.cc:330] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: stopping tablet replica
I20260805 22:46:13.256714 18714 raft_consensus.cc:2229] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:46:13.256718 18712 raft_consensus.cc:2258] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:13.256848 18714 raft_consensus.cc:2258] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:13.257167 18712 ts_tablet_manager.cc:1905] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:13.257433 18714 ts_tablet_manager.cc:1905] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:13.257761 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 18055
I20260805 22:46:13.271992 18712 ts_tablet_manager.cc:1918] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:13.272193 18712 log.cc:1196] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/df3a27db8a294040998cbc713bbd3178
I20260805 22:46:13.272574 18712 ts_tablet_manager.cc:1939] T df3a27db8a294040998cbc713bbd3178 P 8ce78ba3120b432c8b32e1bd218a9ea7: Deleting consensus metadata
I20260805 22:46:13.273913 17954 catalog_manager.cc:4841] TS 8ce78ba3120b432c8b32e1bd218a9ea7 (127.23.197.67:44895): tablet df3a27db8a294040998cbc713bbd3178 (table test-table [id=eb71233b817049a58db17d59386fcd84]) successfully deleted
I20260805 22:46:13.274385 18714 ts_tablet_manager.cc:1918] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:13.274489 18714 log.cc:1196] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/df3a27db8a294040998cbc713bbd3178
I20260805 22:46:13.274858 18714 ts_tablet_manager.cc:1939] T df3a27db8a294040998cbc713bbd3178 P e49032ee96ca42f1982a29f221bd5c02: Deleting consensus metadata
I20260805 22:46:13.275424 17959 catalog_manager.cc:4841] TS e49032ee96ca42f1982a29f221bd5c02 (127.23.197.66:42983): tablet df3a27db8a294040998cbc713bbd3178 (table test-table [id=eb71233b817049a58db17d59386fcd84]) successfully deleted
W20260805 22:46:13.276186 17960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:34177 (error 108)
I20260805 22:46:13.276893 18669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:13.276683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:34177 (local address 127.0.0.1:47518)
0805 22:46:13.276793 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:13.276852 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:34177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260805 22:46:13.276935 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 18258
W20260805 22:46:13.286769 17960 catalog_manager.cc:4584] TS 350db174883741ecaad42777888ed9ba (127.23.197.65:34177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df3a27db8a294040998cbc713bbd3178: Network error: Client connection negotiation failed: client connection to 127.23.197.65:34177: connect: Connection refused (error 111)
I20260805 22:46:13.287035 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 18459
I20260805 22:46:13.299497 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 17929
2026-08-05T22:46:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (817 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:13 dist-test-slave-g170 krb5kdc[18725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:13 dist-test-slave-g170 krb5kdc[18725](info): set up 2 sockets
Aug 05 22:46:13 dist-test-slave-g170 krb5kdc[18725](info): commencing operation
krb5kdc: starting...
W20260805 22:46:14.634433 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:14 dist-test-slave-g170 krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:14.763233 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:42271
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:42271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:14.851909 18743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:14.852180 18743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:14.852223 18743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:14.853467 18743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:14.853543 18743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:14.853570 18743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:14.853592 18743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:14.853613 18743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:14.855235 18743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:42271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:14.855538 18743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:14.855789 18743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:14.861274 18754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:14.861274 18756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:14.861258 18753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:14.861303 18743 server_base.cc:1034] running on GCE node
I20260805 22:46:14.861866 18743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:14.862175 18743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:14.863348 18743 hybrid_clock.cc:648] HybridClock initialized: now 1785969974863353 us; error 47 us; skew 500 ppm
Aug 05 22:46:14 dist-test-slave-g170 krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969974, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:14.866042 18743 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:14.866483 18743 webserver.cc:460] Webserver started at http://127.23.197.126:42851/ using document root <none> and password file <none>
I20260805 22:46:14.866729 18743 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:14.866803 18743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:14.866954 18743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:14.868013 18743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "3835f09209ed4000ad0401a02479fa20"
format_stamp: "Formatted at 2026-08-05 22:46:14 on dist-test-slave-g170"
server_key: "15d4d86a051e1d578f19c12bc7983a11"
server_key_iv: "c7b8ee3b51277d33dc92abee0a8cfe7d"
server_key_version: "encryptionkey@0"
I20260805 22:46:14.868387 18743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "3835f09209ed4000ad0401a02479fa20"
format_stamp: "Formatted at 2026-08-05 22:46:14 on dist-test-slave-g170"
server_key: "15d4d86a051e1d578f19c12bc7983a11"
server_key_iv: "c7b8ee3b51277d33dc92abee0a8cfe7d"
server_key_version: "encryptionkey@0"
I20260805 22:46:14.870016 18743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:14.870872 18766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:14.871054 18743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:14.871162 18743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "3835f09209ed4000ad0401a02479fa20"
format_stamp: "Formatted at 2026-08-05 22:46:14 on dist-test-slave-g170"
server_key: "15d4d86a051e1d578f19c12bc7983a11"
server_key_iv: "c7b8ee3b51277d33dc92abee0a8cfe7d"
server_key_version: "encryptionkey@0"
I20260805 22:46:14.871258 18743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:14.884473 18743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:14.886586 18743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:14.886744 18743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:14.891163 18743 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:42271
I20260805 22:46:14.891314 18837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:42271 every 8 connection(s)
I20260805 22:46:14.891610 18743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:14.892719 18838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:14.895514 18838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20: Bootstrap starting.
I20260805 22:46:14.896232 18838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:14.896595 18838 log.cc:824] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:14.897490 18838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20: No bootstrap required, opened a new log
I20260805 22:46:14.899277 18838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3835f09209ed4000ad0401a02479fa20" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42271 } }
I20260805 22:46:14.899449 18838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:14.899535 18838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3835f09209ed4000ad0401a02479fa20, State: Initialized, Role: FOLLOWER
I20260805 22:46:14.899719 18838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3835f09209ed4000ad0401a02479fa20" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42271 } }
I20260805 22:46:14.899791 18838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:14.899832 18838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:14.899807 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 18743
I20260805 22:46:14.899883 18838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:14.899945 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:14.900154 24341 external_mini_cluster.cc:1428] Setting key 3ffef2402f34377da533eb01edb2103b
I20260805 22:46:14.900559 18838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3835f09209ed4000ad0401a02479fa20" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42271 } }
I20260805 22:46:14.900722 18838 leader_election.cc:302] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3835f09209ed4000ad0401a02479fa20; no voters:
I20260805 22:46:14.900974 18838 leader_election.cc:288] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:14.901010 18845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:14.901211 18845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [term 1 LEADER]: Becoming Leader. State: Replica: 3835f09209ed4000ad0401a02479fa20, State: Running, Role: LEADER
I20260805 22:46:14.901347 18838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:14.901371 18845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3835f09209ed4000ad0401a02479fa20" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42271 } }
I20260805 22:46:14.901870 18845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3835f09209ed4000ad0401a02479fa20. Latest consensus state: current_term: 1 leader_uuid: "3835f09209ed4000ad0401a02479fa20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3835f09209ed4000ad0401a02479fa20" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42271 } } }
I20260805 22:46:14.902001 18845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:14.902055 18847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3835f09209ed4000ad0401a02479fa20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3835f09209ed4000ad0401a02479fa20" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42271 } } }
I20260805 22:46:14.902158 18847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:14.902268 18860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:14.902909 18860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:14.904897 18860 catalog_manager.cc:1347] Generated new cluster ID: 9295d89643c9435980d12b70589cf560
I20260805 22:46:14.904958 18860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:46:14 dist-test-slave-g170 krb5kdc[18725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:14.911466 18860 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:14.912072 18860 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:14.913733 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:14.900800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52852 (local address 127.23.197.126:42271)
0805 22:46:14.900991 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:14.901002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:14.901117 (+ 115us) server_negotiation.cc:407] Connection header received
0805 22:46:14.901258 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:14.901268 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:14.901336 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:14.902484 (+ 1148us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:14.902626 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:14.904140 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:14.905242 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:14.905903 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:14.906013 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:14.909101 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:14.909105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:14.909106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:14.909119 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:14.911912 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:14.912369 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:14.912372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:14.912373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:14.912488 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:14.912735 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:14.912736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:14.912737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:14.913061 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:14.913233 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:14.913345 (+ 112us) server_negotiation.cc:299] Negotiation successful
0805 22:46:14.913501 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260805 22:46:14.915895 18860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20: Generated new TSK 0
I20260805 22:46:14.916101 18860 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:46:14.948060 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:42271
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:15.046907 18876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:15.047134 18876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:15.047163 18876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:15.048457 18876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:15.048509 18876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:15.048540 18876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:46:15.050235 18876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:42271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:15.050510 18876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:15.050784 18876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:15.056795 18883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:15.056700 18887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:15.056686 18884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:15.056735 18876 server_base.cc:1034] running on GCE node
I20260805 22:46:15.057178 18876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:15.057452 18876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:15.058604 18876 hybrid_clock.cc:648] HybridClock initialized: now 1785969975058578 us; error 47 us; skew 500 ppm
Aug 05 22:46:15 dist-test-slave-g170 krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969975, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:15.061201 18876 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:46:15.061628 18876 webserver.cc:460] Webserver started at http://127.23.197.65:39065/ using document root <none> and password file <none>
I20260805 22:46:15.061877 18876 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:15.061949 18876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:15.062103 18876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:15.063104 18876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "2658ec517c50415e924f35264ac828a0"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "2e5f524cbfbc7645c965e6ffc23d675a"
server_key_iv: "a83487552953a5fadd767999aef89574"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.063467 18876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "2658ec517c50415e924f35264ac828a0"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "2e5f524cbfbc7645c965e6ffc23d675a"
server_key_iv: "a83487552953a5fadd767999aef89574"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.064914 18876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260805 22:46:15.065758 18898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.065991 18876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:15.066097 18876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "2658ec517c50415e924f35264ac828a0"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "2e5f524cbfbc7645c965e6ffc23d675a"
server_key_iv: "a83487552953a5fadd767999aef89574"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.066176 18876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:15.075217 18876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:15.077098 18876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:15.077257 18876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:15.077488 18876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:15.077811 18876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:15.077864 18876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.077898 18876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:15.077925 18876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.083838 18876 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:44177
I20260805 22:46:15.083884 19063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:44177 every 8 connection(s)
I20260805 22:46:15.084193 18876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:46:15.085161 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 18876
I20260805 22:46:15.085297 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:46:15.085489 24341 external_mini_cluster.cc:1428] Setting key 0475786695965c6fe34fccd5e8174d70
Aug 05 22:46:15 dist-test-slave-g170 krb5kdc[18725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969975, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
I20260805 22:46:15.095010 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.084999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:44531 (local address 127.23.197.126:42271)
0805 22:46:15.085061 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.085063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.085460 (+ 397us) server_negotiation.cc:407] Connection header received
0805 22:46:15.086097 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.086099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.086111 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.086159 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:15.087123 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.087640 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.088643 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.088985 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.089058 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.091636 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:15.091638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:15.091638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:15.091644 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:15.093258 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:15.093785 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:15.093786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:15.093787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:15.093848 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:15.094161 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:15.094162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:15.094162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:15.094281 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:15.094339 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.094777 (+ 438us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.094896 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:46:15.095093 19068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.085103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:42271 (local address 127.23.197.65:44531)
0805 22:46:15.085309 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:15.085359 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:15.085989 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:15.086227 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:15.086231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:15.086813 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:15.087001 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.087005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.087739 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.087740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.088547 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.088549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.089080 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.089081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.089191 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.089735 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:15.089748 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:15.091538 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:15.093363 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:15.093370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:15.093373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:15.093684 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:15.093930 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:15.093931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:15.093932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:15.094081 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:15.094353 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:15.094360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:15.094539 (+ 179us) client_negotiation.cc:769] Sending connection context
0805 22:46:15.094655 (+ 116us) client_negotiation.cc:240] Negotiation successful
0805 22:46:15.094835 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":91,"threads_started":1}
I20260805 22:46:15.095850 19066 heartbeater.cc:347] Connected to a master server at 127.23.197.126:42271
I20260805 22:46:15.095963 19066 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:15.096138 19066 heartbeater.cc:511] Master 127.23.197.126:42271 requested a full tablet report, sending...
I20260805 22:46:15.096690 18786 ts_manager.cc:194] Registered new tserver with Master: 2658ec517c50415e924f35264ac828a0 (127.23.197.65:44177)
I20260805 22:46:15.097534 18786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:44531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:46:15.112126 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:42271
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:15.199451 19073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:15.199695 19073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:15.199744 19073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:15.200992 19073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:15.201056 19073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:15.201100 19073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:46:15.202713 19073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:42271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:15.202987 19073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:15.203224 19073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:15.208432 19079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:15.208406 19080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:15.208462 19083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:15.208519 19073 server_base.cc:1034] running on GCE node
I20260805 22:46:15.208832 19073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:15.209049 19073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:15.210196 19073 hybrid_clock.cc:648] HybridClock initialized: now 1785969975210177 us; error 38 us; skew 500 ppm
Aug 05 22:46:15 dist-test-slave-g170 krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969975, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:15.212695 19073 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:46:15.213183 19073 webserver.cc:460] Webserver started at http://127.23.197.66:39161/ using document root <none> and password file <none>
I20260805 22:46:15.213424 19073 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:15.213493 19073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:15.213649 19073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:15.214704 19073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "43217f674e7a490b8419a56c84528934"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "1f20450028623517414de03abc85419a"
server_key_iv: "f96e7b322b8068bc40ca88b468131190"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.215075 19073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "43217f674e7a490b8419a56c84528934"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "1f20450028623517414de03abc85419a"
server_key_iv: "f96e7b322b8068bc40ca88b468131190"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.216451 19073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:15.217269 19090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.217463 19073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:46:15.217572 19073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "43217f674e7a490b8419a56c84528934"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "1f20450028623517414de03abc85419a"
server_key_iv: "f96e7b322b8068bc40ca88b468131190"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.217649 19073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:15.243397 19073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:15.245831 19073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:15.245983 19073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:15.246258 19073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:15.246579 19073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:15.246635 19073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.246670 19073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:15.246688 19073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.252561 19073 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:46673
I20260805 22:46:15.252620 19263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:46673 every 8 connection(s)
I20260805 22:46:15.252981 19073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:46:15.258039 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 19073
I20260805 22:46:15.258173 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:46:15.258373 24341 external_mini_cluster.cc:1428] Setting key 350a6f2a02481f3d6b67ca1096af6bb0
Aug 05 22:46:15 dist-test-slave-g170 krb5kdc[18725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969975, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:15.263710 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.253759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:57461 (local address 127.23.197.126:42271)
0805 22:46:15.253833 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.253835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.254036 (+ 201us) server_negotiation.cc:407] Connection header received
0805 22:46:15.254514 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.254515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.254526 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.254590 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:15.255570 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.256106 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.257369 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.257717 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.257778 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.260464 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:15.260465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:15.260466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:15.260474 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:15.262175 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:15.262739 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:15.262740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:15.262740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:15.262794 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:15.263096 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:15.263097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:15.263097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:15.263197 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:15.263248 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.263568 (+ 320us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.263638 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 22:46:15.264128 19266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.253717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:42271 (local address 127.23.197.66:57461)
0805 22:46:15.253911 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:15.253948 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:15.254426 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:15.254675 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:15.254680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:15.255264 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:15.255461 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.255466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.256209 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.256210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.257271 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.257274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.257833 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.257834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.257967 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.258571 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:15.258580 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:15.260365 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:15.262268 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:15.262277 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:15.262279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:15.262632 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:15.262864 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:15.262865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:15.262866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:15.263030 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:15.263262 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:15.263268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:15.263433 (+ 165us) client_negotiation.cc:769] Sending connection context
0805 22:46:15.263548 (+ 115us) client_negotiation.cc:240] Negotiation successful
0805 22:46:15.263725 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
I20260805 22:46:15.264758 19264 heartbeater.cc:347] Connected to a master server at 127.23.197.126:42271
I20260805 22:46:15.264889 19264 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:15.265085 19264 heartbeater.cc:511] Master 127.23.197.126:42271 requested a full tablet report, sending...
I20260805 22:46:15.265496 18786 ts_manager.cc:194] Registered new tserver with Master: 43217f674e7a490b8419a56c84528934 (127.23.197.66:46673)
I20260805 22:46:15.265894 18786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:57461
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:46:15.288607 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:42271
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:15.376431 19270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:15.376637 19270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:15.376677 19270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:15.377864 19270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:15.377907 19270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:15.377933 19270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:46:15.379369 19270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:42271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:15.379571 19270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:15.379770 19270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:15.384965 19281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:15.384984 19277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:15.384991 19278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:15.385275 19270 server_base.cc:1034] running on GCE node
I20260805 22:46:15.385502 19270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:15.385762 19270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:15.386890 19270 hybrid_clock.cc:648] HybridClock initialized: now 1785969975386866 us; error 38 us; skew 500 ppm
Aug 05 22:46:15 dist-test-slave-g170 krb5kdc[18725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969975, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:15.390208 19270 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:46:15.390677 19270 webserver.cc:460] Webserver started at http://127.23.197.67:33225/ using document root <none> and password file <none>
I20260805 22:46:15.390951 19270 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:15.391023 19270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:15.391183 19270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:15.392261 19270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "a7ea87f40dcc49f2848beed5840a8aba"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "6a46c63673e0cc8e60400e60f934b2ab"
server_key_iv: "87a1d9920fef3a7f777bb0a7d36c85ee"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.392630 19270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "a7ea87f40dcc49f2848beed5840a8aba"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "6a46c63673e0cc8e60400e60f934b2ab"
server_key_iv: "87a1d9920fef3a7f777bb0a7d36c85ee"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.394186 19270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:15.395066 19290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.395295 19270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:15.395407 19270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "a7ea87f40dcc49f2848beed5840a8aba"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "6a46c63673e0cc8e60400e60f934b2ab"
server_key_iv: "87a1d9920fef3a7f777bb0a7d36c85ee"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.395510 19270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:15.416852 19270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:15.419454 19270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:15.419648 19270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:15.419989 19270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:15.420437 19270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:15.420500 19270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.420589 19270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:15.420626 19270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.427872 19270 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:38669
I20260805 22:46:15.427933 19455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:38669 every 8 connection(s)
I20260805 22:46:15.428336 19270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:46:15.434911 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 19270
I20260805 22:46:15.435009 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
Aug 05 22:46:15 dist-test-slave-g170 krb5kdc[18725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969975, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:15.435218 24341 external_mini_cluster.cc:1428] Setting key 406cec1c59cae6a44a6a244ad31e9881
I20260805 22:46:15.438771 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.429293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:39543 (local address 127.23.197.126:42271)
0805 22:46:15.429386 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.429389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.429598 (+ 209us) server_negotiation.cc:407] Connection header received
0805 22:46:15.430128 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.430130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.430141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.430211 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:15.431190 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.431711 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.432932 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.433166 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.433237 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.435819 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:15.435820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:15.435821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:15.435828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:15.437310 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:15.437745 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:15.437748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:15.437749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:15.437807 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:15.438075 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:15.438076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:15.438076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:15.438199 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:15.438282 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.438586 (+ 304us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.438673 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 22:46:15.438961 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.429224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:42271 (local address 127.23.197.67:39543)
0805 22:46:15.429431 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:15.429475 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:15.430039 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:15.430269 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:15.430273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:15.430861 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:15.431079 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.431083 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.431812 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.431813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.432638 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.432640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.433257 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.433258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.433388 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.433993 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:15.434002 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:15.435725 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:15.437409 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:15.437416 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:15.437418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:15.437654 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:15.437876 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:15.437877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:15.437878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:15.437997 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:15.438271 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:15.438279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:15.438440 (+ 161us) client_negotiation.cc:769] Sending connection context
0805 22:46:15.438550 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 22:46:15.438717 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260805 22:46:15.439435 19456 heartbeater.cc:347] Connected to a master server at 127.23.197.126:42271
I20260805 22:46:15.439543 19456 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:15.439776 19456 heartbeater.cc:511] Master 127.23.197.126:42271 requested a full tablet report, sending...
I20260805 22:46:15.440236 18786 ts_manager.cc:194] Registered new tserver with Master: a7ea87f40dcc49f2848beed5840a8aba (127.23.197.67:38669)
I20260805 22:46:15.440642 18786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:39543
I20260805 22:46:15.447417 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:46:15.455798 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.448947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.23.197.126:42271)
0805 22:46:15.449019 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.449021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.449113 (+ 92us) server_negotiation.cc:407] Connection header received
0805 22:46:15.449208 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.449209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.449218 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.449269 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:15.450073 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.450594 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.451391 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.451587 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.451647 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.453104 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:15.453106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:15.453106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:15.453112 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:15.454342 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:15.454769 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:15.454770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:15.454771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:15.454832 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:15.455044 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:15.455045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:15.455045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:15.455192 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:15.455293 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.455505 (+ 212us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.455567 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 22:46:15.457890 18786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:46:15.459656 18786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:46:15.467283 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.462422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:38669 (local address 127.0.0.1:51592)
0805 22:46:15.462546 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:15.462586 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:15.462648 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:15.463097 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:15.463101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:15.463103 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:15.463213 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.463217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.464614 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.464615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.466164 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.466166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.466932 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.466933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.467023 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.467038 (+ 15us) client_negotiation.cc:769] Sending connection context
0805 22:46:15.467131 (+ 93us) client_negotiation.cc:240] Negotiation successful
0805 22:46:15.467194 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":76,"threads_started":1}
I20260805 22:46:15.467298 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.462540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.23.197.67:38669)
0805 22:46:15.462887 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.462893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.462906 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:46:15.462951 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.462954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.462974 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.463083 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:15.463316 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.464501 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.466294 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.466831 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.466933 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.466998 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.467164 (+ 166us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.467211 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":124,"threads_started":1}
I20260805 22:46:15.467603 19475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.462808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.23.197.65:44177)
0805 22:46:15.463010 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.463014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.463209 (+ 195us) server_negotiation.cc:407] Connection header received
0805 22:46:15.463354 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.463357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.463377 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.463459 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:15.463576 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.464628 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.466244 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.466973 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.467044 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.467095 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.467477 (+ 382us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.467523 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":89,"threads_started":1}
I20260805 22:46:15.467401 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.462659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:44177 (local address 127.0.0.1:51844)
0805 22:46:15.463069 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:15.463080 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:15.463253 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:15.463455 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:15.463456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:15.463458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:15.463509 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.463511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.464801 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.464803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.466140 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.466144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.467131 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.467133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.467240 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.467244 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:15.467289 (+ 45us) client_negotiation.cc:240] Negotiation successful
0805 22:46:15.467332 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":79,"threads_started":1}
I20260805 22:46:15.468374 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.462862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:46673 (local address 127.0.0.1:43384)
0805 22:46:15.463167 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:15.463177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:15.463227 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:15.464886 (+ 1659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:15.464887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:15.464889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:15.464965 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.464967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.466017 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.466018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.467115 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.467119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.468141 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.468143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.468204 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.468209 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:46:15.468261 (+ 52us) client_negotiation.cc:240] Negotiation successful
0805 22:46:15.468300 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":27,"thread_start_us":75,"threads_started":1}
I20260805 22:46:15.468626 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.462980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43384 (local address 127.23.197.66:46673)
0805 22:46:15.464695 (+ 1715us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.464699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.464710 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:46:15.464749 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.464751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.464788 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.464887 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:15.465030 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.465914 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.467208 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.467971 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.468048 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.468106 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.468481 (+ 375us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.468529 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1671,"thread_start_us":97,"threads_started":1}
I20260805 22:46:15.468640 19362 tablet_service.cc:1467] Processing CreateTablet for tablet b065c00b31034bd4bb9ce967f943417e (DEFAULT_TABLE table=test-table [id=4983cbdf0d8d4131be8719473b8fd51f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:15.468752 18967 tablet_service.cc:1467] Processing CreateTablet for tablet b065c00b31034bd4bb9ce967f943417e (DEFAULT_TABLE table=test-table [id=4983cbdf0d8d4131be8719473b8fd51f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:15.469102 19362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b065c00b31034bd4bb9ce967f943417e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:15.469105 18967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b065c00b31034bd4bb9ce967f943417e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:15.469238 19165 tablet_service.cc:1467] Processing CreateTablet for tablet b065c00b31034bd4bb9ce967f943417e (DEFAULT_TABLE table=test-table [id=4983cbdf0d8d4131be8719473b8fd51f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:15.469563 19165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b065c00b31034bd4bb9ce967f943417e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:15.472055 19481 tablet_bootstrap.cc:492] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: Bootstrap starting.
I20260805 22:46:15.472805 19481 tablet_bootstrap.cc:654] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:15.472780 19483 tablet_bootstrap.cc:492] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: Bootstrap starting.
I20260805 22:46:15.473096 19482 tablet_bootstrap.cc:492] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0: Bootstrap starting.
I20260805 22:46:15.473426 19481 log.cc:824] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:15.473852 19482 tablet_bootstrap.cc:654] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:15.474326 19481 tablet_bootstrap.cc:492] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: No bootstrap required, opened a new log
I20260805 22:46:15.474388 19481 ts_tablet_manager.cc:1397] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:46:15.474419 19482 log.cc:824] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:15.475324 19483 tablet_bootstrap.cc:654] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:15.475315 19482 tablet_bootstrap.cc:492] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0: No bootstrap required, opened a new log
I20260805 22:46:15.475411 19482 ts_tablet_manager.cc:1397] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:46:15.475849 19483 log.cc:824] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:15.476824 19483 tablet_bootstrap.cc:492] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: No bootstrap required, opened a new log
I20260805 22:46:15.476580 19481 raft_consensus.cc:348] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } }
I20260805 22:46:15.476873 19481 raft_consensus.cc:374] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:15.476897 19483 ts_tablet_manager.cc:1397] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 22:46:15.476930 19481 raft_consensus.cc:729] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43217f674e7a490b8419a56c84528934, State: Initialized, Role: FOLLOWER
I20260805 22:46:15.477066 19481 consensus_queue.cc:260] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } }
I20260805 22:46:15.477377 19481 ts_tablet_manager.cc:1428] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:15.477411 19264 heartbeater.cc:503] Master 127.23.197.126:42271 was elected leader, sending a full tablet report...
I20260805 22:46:15.477941 19482 raft_consensus.cc:348] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } }
I20260805 22:46:15.478106 19482 raft_consensus.cc:374] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:15.478154 19482 raft_consensus.cc:729] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2658ec517c50415e924f35264ac828a0, State: Initialized, Role: FOLLOWER
I20260805 22:46:15.478348 19482 consensus_queue.cc:260] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } }
I20260805 22:46:15.478708 19066 heartbeater.cc:503] Master 127.23.197.126:42271 was elected leader, sending a full tablet report...
I20260805 22:46:15.478708 19482 ts_tablet_manager.cc:1428] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:15.478986 19483 raft_consensus.cc:348] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } }
I20260805 22:46:15.479166 19483 raft_consensus.cc:374] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:15.479238 19483 raft_consensus.cc:729] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ea87f40dcc49f2848beed5840a8aba, State: Initialized, Role: FOLLOWER
I20260805 22:46:15.479364 19483 consensus_queue.cc:260] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } }
I20260805 22:46:15.479650 19483 ts_tablet_manager.cc:1428] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260805 22:46:15.479702 19456 heartbeater.cc:503] Master 127.23.197.126:42271 was elected leader, sending a full tablet report...
W20260805 22:46:15.503558 19265 tablet.cc:2369] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 22:46:15.585390 19067 tablet.cc:2369] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:46:15.648993 19490 raft_consensus.cc:482] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:46:15.649150 19490 raft_consensus.cc:504] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } }
I20260805 22:46:15.649554 19490 leader_election.cc:288] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7ea87f40dcc49f2848beed5840a8aba (127.23.197.67:38669), 43217f674e7a490b8419a56c84528934 (127.23.197.66:46673)
I20260805 22:46:15.652665 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.649899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:38655 (local address 127.23.197.67:38669)
0805 22:46:15.649946 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.649948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.650074 (+ 126us) server_negotiation.cc:407] Connection header received
0805 22:46:15.650108 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.650109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.650120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.650216 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:15.650414 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.650912 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.651894 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.652291 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.652376 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.652386 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.652558 (+ 172us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.652601 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:46:15.652680 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.649819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:38669 (local address 127.23.197.65:38655)
0805 22:46:15.649950 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:15.649963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:15.650029 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:15.650240 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:15.650242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:15.650246 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:15.650333 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.650336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.651008 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.651009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.651795 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.651797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.652386 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.652388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.652463 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.652472 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 22:46:15.652538 (+ 66us) client_negotiation.cc:240] Negotiation successful
0805 22:46:15.652580 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260805 22:46:15.652738 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.650091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:55339 (local address 127.23.197.66:46673)
0805 22:46:15.650154 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.650156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.650190 (+ 34us) server_negotiation.cc:407] Connection header received
0805 22:46:15.650254 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.650255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.650266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.650338 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:15.650540 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.651002 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.651953 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.652344 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.652433 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.652441 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.652601 (+ 160us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.652642 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260805 22:46:15.653003 19494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.650022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:46673 (local address 127.23.197.65:55339)
0805 22:46:15.650136 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:15.650147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:15.650200 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:15.650341 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:15.650342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:15.650343 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:15.650449 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.650451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.651084 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.651085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.651856 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:15.651858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.652432 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.652433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:15.652511 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.652514 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:15.652601 (+ 87us) client_negotiation.cc:240] Negotiation successful
0805 22:46:15.652642 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":41,"threads_started":1}
I20260805 22:46:15.653210 19388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b065c00b31034bd4bb9ce967f943417e" candidate_uuid: "2658ec517c50415e924f35264ac828a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ea87f40dcc49f2848beed5840a8aba" is_pre_election: true
I20260805 22:46:15.653385 19388 raft_consensus.cc:2454] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2658ec517c50415e924f35264ac828a0 in term 0.
I20260805 22:46:15.653625 18900 leader_election.cc:302] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2658ec517c50415e924f35264ac828a0, a7ea87f40dcc49f2848beed5840a8aba; no voters:
I20260805 22:46:15.653719 19192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b065c00b31034bd4bb9ce967f943417e" candidate_uuid: "2658ec517c50415e924f35264ac828a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43217f674e7a490b8419a56c84528934" is_pre_election: true
I20260805 22:46:15.653748 19490 raft_consensus.cc:2781] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:46:15.653810 19490 raft_consensus.cc:482] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:46:15.653844 19490 raft_consensus.cc:3037] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:15.653873 19192 raft_consensus.cc:2454] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2658ec517c50415e924f35264ac828a0 in term 0.
I20260805 22:46:15.654627 19490 raft_consensus.cc:504] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } }
I20260805 22:46:15.654767 19490 leader_election.cc:288] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [CANDIDATE]: Term 1 election: Requested vote from peers a7ea87f40dcc49f2848beed5840a8aba (127.23.197.67:38669), 43217f674e7a490b8419a56c84528934 (127.23.197.66:46673)
I20260805 22:46:15.654974 19388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b065c00b31034bd4bb9ce967f943417e" candidate_uuid: "2658ec517c50415e924f35264ac828a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ea87f40dcc49f2848beed5840a8aba"
I20260805 22:46:15.655043 19388 raft_consensus.cc:3037] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:15.655822 19388 raft_consensus.cc:2454] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2658ec517c50415e924f35264ac828a0 in term 1.
I20260805 22:46:15.655941 19192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b065c00b31034bd4bb9ce967f943417e" candidate_uuid: "2658ec517c50415e924f35264ac828a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43217f674e7a490b8419a56c84528934"
I20260805 22:46:15.655989 18900 leader_election.cc:302] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2658ec517c50415e924f35264ac828a0, a7ea87f40dcc49f2848beed5840a8aba; no voters:
I20260805 22:46:15.656076 19490 raft_consensus.cc:2781] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:15.656030 19192 raft_consensus.cc:3037] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:15.656229 19490 raft_consensus.cc:686] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [term 1 LEADER]: Becoming Leader. State: Replica: 2658ec517c50415e924f35264ac828a0, State: Running, Role: LEADER
I20260805 22:46:15.656349 19490 consensus_queue.cc:237] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } }
I20260805 22:46:15.657024 19192 raft_consensus.cc:2454] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2658ec517c50415e924f35264ac828a0 in term 1.
I20260805 22:46:15.657255 18786 catalog_manager.cc:5495] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2658ec517c50415e924f35264ac828a0 (127.23.197.65). New cstate: current_term: 1 leader_uuid: "2658ec517c50415e924f35264ac828a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2658ec517c50415e924f35264ac828a0" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 44177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 } health_report { overall_health: UNKNOWN } } }
W20260805 22:46:15.679071 19458 tablet.cc:2369] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:46:15.696190 19475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.693695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.23.197.65:44177)
0805 22:46:15.693796 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.693798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.693809 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:46:15.693886 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.693887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.693905 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.693980 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:15.694143 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.694618 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.695389 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.695581 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.695652 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.695789 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:15.695882 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:15.695957 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.696112 (+ 155us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.696130 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260805 22:46:15.698421 19388 raft_consensus.cc:1261] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [term 1 FOLLOWER]: Refusing update from remote peer 2658ec517c50415e924f35264ac828a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:15.698421 19192 raft_consensus.cc:1261] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [term 1 FOLLOWER]: Refusing update from remote peer 2658ec517c50415e924f35264ac828a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:15.698823 19490 consensus_queue.cc:1035] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43217f674e7a490b8419a56c84528934" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 46673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:15.698997 19495 consensus_queue.cc:1035] T b065c00b31034bd4bb9ce967f943417e P 2658ec517c50415e924f35264ac828a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7ea87f40dcc49f2848beed5840a8aba" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:15.701830 19501 mvcc.cc:205] Tried to move back new op lower bound from 7315333020457238528 to 7315333020289015808. Current Snapshot: MvccSnapshot[applied={T|T < 7315333020457238528}]
I20260805 22:46:15.702479 19500 mvcc.cc:205] Tried to move back new op lower bound from 7315333020457238528 to 7315333020289015808. Current Snapshot: MvccSnapshot[applied={T|T < 7315333020457238528}]
I20260805 22:46:15.705148 18786 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:15.705246 18786 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:15.706066 18786 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3835f09209ed4000ad0401a02479fa20: Sending DeleteTablet for 3 replicas of tablet b065c00b31034bd4bb9ce967f943417e
I20260805 22:46:15.706387 19362 tablet_service.cc:1514] Processing DeleteTablet for tablet b065c00b31034bd4bb9ce967f943417e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:15 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:51592
I20260805 22:46:15.706446 18967 tablet_service.cc:1514] Processing DeleteTablet for tablet b065c00b31034bd4bb9ce967f943417e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:15 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:51844
I20260805 22:46:15.706482 19165 tablet_service.cc:1514] Processing DeleteTablet for tablet b065c00b31034bd4bb9ce967f943417e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:15 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:43384
I20260805 22:46:15.706664 19510 tablet_replica.cc:330] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: stopping tablet replica
I20260805 22:46:15.706771 19510 raft_consensus.cc:2229] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:46:15.706880 19510 raft_consensus.cc:2258] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:15.707105 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 18876
I20260805 22:46:15.714401 19510 ts_tablet_manager.cc:1905] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:15.716902 19510 ts_tablet_manager.cc:1918] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:15.717001 19510 log.cc:1196] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/b065c00b31034bd4bb9ce967f943417e
I20260805 22:46:15.717383 19510 ts_tablet_manager.cc:1939] T b065c00b31034bd4bb9ce967f943417e P a7ea87f40dcc49f2848beed5840a8aba: Deleting consensus metadata
I20260805 22:46:15.718101 18768 catalog_manager.cc:4841] TS a7ea87f40dcc49f2848beed5840a8aba (127.23.197.67:38669): tablet b065c00b31034bd4bb9ce967f943417e (table test-table [id=4983cbdf0d8d4131be8719473b8fd51f]) successfully deleted
I20260805 22:46:15.723984 19514 tablet_replica.cc:330] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: stopping tablet replica
I20260805 22:46:15.724155 19514 raft_consensus.cc:2229] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:46:15.724306 19514 raft_consensus.cc:2258] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:15.724880 19514 ts_tablet_manager.cc:1905] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:15.726867 19514 ts_tablet_manager.cc:1918] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:15.726953 19514 log.cc:1196] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/b065c00b31034bd4bb9ce967f943417e
I20260805 22:46:15.727209 19514 ts_tablet_manager.cc:1939] T b065c00b31034bd4bb9ce967f943417e P 43217f674e7a490b8419a56c84528934: Deleting consensus metadata
I20260805 22:46:15.727677 18768 catalog_manager.cc:4841] TS 43217f674e7a490b8419a56c84528934 (127.23.197.66:46673): tablet b065c00b31034bd4bb9ce967f943417e (table test-table [id=4983cbdf0d8d4131be8719473b8fd51f]) successfully deleted
W20260805 22:46:15.729619 18768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:44177 (error 108)
I20260805 22:46:15.729823 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 19073
I20260805 22:46:15.735252 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.735084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:44177 (local address 127.0.0.1:51858)
0805 22:46:15.735159 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:15.735210 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260805 22:46:15.735378 18768 catalog_manager.cc:4584] TS 2658ec517c50415e924f35264ac828a0 (127.23.197.65:44177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b065c00b31034bd4bb9ce967f943417e: Network error: Client connection negotiation failed: client connection to 127.23.197.65:44177: connect: Connection refused (error 111)
I20260805 22:46:15.745373 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 19270
I20260805 22:46:15.754297 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 18743
2026-08-05T22:46:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2456 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-05T22:46:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:15Z Disabled control of system clock
I20260805 22:46:15.790194 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33441
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:45707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:33441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 22:46:15.874001 19519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:15.874212 19519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:15.874233 19519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:15.875392 19519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:15.875435 19519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:15.875448 19519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:15.875460 19519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:15.875514 19519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:15.876932 19519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:33441
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:15.877202 19519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:15.877544 19519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:15.882428 19529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:15.882475 19530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:15.882486 19536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:15.882911 19519 server_base.cc:1034] running on GCE node
I20260805 22:46:15.883173 19519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:15.883469 19519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:15.884685 19519 hybrid_clock.cc:648] HybridClock initialized: now 1785969975884664 us; error 50 us; skew 500 ppm
I20260805 22:46:15.885955 19519 webserver.cc:460] Webserver started at http://127.23.197.126:36977/ using document root <none> and password file <none>
I20260805 22:46:15.886204 19519 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:15.886277 19519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:15.886430 19519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:15.887533 19519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "386fb95c0a904372a3799fa2ecadea09"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "b825df18a4f223504fb1bdcdf87ab592"
server_key_iv: "dae06d86590d4837b5f3696842154ed3"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.887931 19519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "386fb95c0a904372a3799fa2ecadea09"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "b825df18a4f223504fb1bdcdf87ab592"
server_key_iv: "dae06d86590d4837b5f3696842154ed3"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.889413 19519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:15.890311 19544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:15.890534 19519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:15.890686 19519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "386fb95c0a904372a3799fa2ecadea09"
format_stamp: "Formatted at 2026-08-05 22:46:15 on dist-test-slave-g170"
server_key: "b825df18a4f223504fb1bdcdf87ab592"
server_key_iv: "dae06d86590d4837b5f3696842154ed3"
server_key_version: "encryptionkey@0"
I20260805 22:46:15.890766 19519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:15.906203 19519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:15.906527 19519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:15.906688 19519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:15.910976 19519 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:33441
I20260805 22:46:15.911023 19628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:33441 every 8 connection(s)
I20260805 22:46:15.911394 19519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:15.912477 19630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:15.914808 19630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09: Bootstrap starting.
I20260805 22:46:15.915371 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 19519
I20260805 22:46:15.915459 19630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:15.915464 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:15.915679 24341 external_mini_cluster.cc:1428] Setting key 920ff5328ed8097a659b97e7d2509fb8
I20260805 22:46:15.915802 19630 log.cc:824] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:15.916677 19630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09: No bootstrap required, opened a new log
I20260805 22:46:15.918898 19630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386fb95c0a904372a3799fa2ecadea09" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33441 } }
I20260805 22:46:15.919064 19630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:15.919095 19630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 386fb95c0a904372a3799fa2ecadea09, State: Initialized, Role: FOLLOWER
I20260805 22:46:15.919219 19630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386fb95c0a904372a3799fa2ecadea09" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33441 } }
I20260805 22:46:15.919289 19630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:15.919317 19630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:15.919373 19630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:15.919951 19630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386fb95c0a904372a3799fa2ecadea09" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33441 } }
I20260805 22:46:15.920068 19630 leader_election.cc:302] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 386fb95c0a904372a3799fa2ecadea09; no voters:
I20260805 22:46:15.920254 19630 leader_election.cc:288] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:15.920317 19639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:15.920452 19639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [term 1 LEADER]: Becoming Leader. State: Replica: 386fb95c0a904372a3799fa2ecadea09, State: Running, Role: LEADER
I20260805 22:46:15.920552 19639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386fb95c0a904372a3799fa2ecadea09" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33441 } }
I20260805 22:46:15.920586 19630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:15.920948 19639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 386fb95c0a904372a3799fa2ecadea09. Latest consensus state: current_term: 1 leader_uuid: "386fb95c0a904372a3799fa2ecadea09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386fb95c0a904372a3799fa2ecadea09" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33441 } } }
I20260805 22:46:15.921043 19639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:15.920957 19640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "386fb95c0a904372a3799fa2ecadea09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386fb95c0a904372a3799fa2ecadea09" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33441 } } }
I20260805 22:46:15.921231 19640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:15.921663 19649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:15.921887 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:15.916250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59644 (local address 127.23.197.126:33441)
0805 22:46:15.916451 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:15.916458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:15.916487 (+ 29us) server_negotiation.cc:407] Connection header received
0805 22:46:15.916960 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:15.916967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:15.917024 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:15.917139 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:15.917393 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.918283 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.919406 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:15.919850 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:15.919945 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:15.920913 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:15.920917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:15.920918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:15.921130 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:15.921171 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:15.921186 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:15.921259 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:15.921494 (+ 235us) server_negotiation.cc:299] Negotiation successful
0805 22:46:15.921590 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260805 22:46:15.922377 19649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:15.925959 19649 catalog_manager.cc:1347] Generated new cluster ID: 9af309ace1cc4e83974276d8c5439f57
I20260805 22:46:15.926029 19649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:15.933211 19649 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:15.933825 19649 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:15.937357 19649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09: Generated new TSK 0
I20260805 22:46:15.937548 19649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:15.938849 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33441
--builtin_ntp_servers=127.23.197.84:45707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 22:46:16.024789 19671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:16.025010 19671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:16.025046 19671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:16.026207 19671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:16.026270 19671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:16.026314 19671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:46:16.027697 19671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:16.027953 19671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:16.028265 19671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:16.034403 19679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:16.034457 19682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:16.034524 19678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:16.034636 19671 server_base.cc:1034] running on GCE node
I20260805 22:46:16.034899 19671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:16.035204 19671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:16.036374 19671 hybrid_clock.cc:648] HybridClock initialized: now 1785969976036354 us; error 48 us; skew 500 ppm
I20260805 22:46:16.037945 19671 webserver.cc:460] Webserver started at http://127.23.197.65:36885/ using document root <none> and password file <none>
I20260805 22:46:16.038236 19671 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:16.038298 19671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:16.038508 19671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:16.039692 19671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "b5da4abeda8e427b97aedf205c3636a1"
format_stamp: "Formatted at 2026-08-05 22:46:16 on dist-test-slave-g170"
server_key: "e53e932e9a8fe0b0ffbe7c0c1a0a7342"
server_key_iv: "53d79fcdb0f20ac376d266bfd558368f"
server_key_version: "encryptionkey@0"
I20260805 22:46:16.040095 19671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "b5da4abeda8e427b97aedf205c3636a1"
format_stamp: "Formatted at 2026-08-05 22:46:16 on dist-test-slave-g170"
server_key: "e53e932e9a8fe0b0ffbe7c0c1a0a7342"
server_key_iv: "53d79fcdb0f20ac376d266bfd558368f"
server_key_version: "encryptionkey@0"
I20260805 22:46:16.041687 19671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260805 22:46:16.042552 19689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:16.042824 19671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:16.042946 19671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "b5da4abeda8e427b97aedf205c3636a1"
format_stamp: "Formatted at 2026-08-05 22:46:16 on dist-test-slave-g170"
server_key: "e53e932e9a8fe0b0ffbe7c0c1a0a7342"
server_key_iv: "53d79fcdb0f20ac376d266bfd558368f"
server_key_version: "encryptionkey@0"
I20260805 22:46:16.043035 19671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:16.065167 19671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:16.065519 19671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:16.065690 19671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:16.065954 19671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:16.066319 19671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:16.066382 19671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:16.066428 19671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:16.066459 19671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:16.073096 19671 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:32807
I20260805 22:46:16.073150 19855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:32807 every 8 connection(s)
I20260805 22:46:16.073544 19671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:46:16.075100 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 19671
I20260805 22:46:16.075237 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:46:16.075480 24341 external_mini_cluster.cc:1428] Setting key cf14b904b0a5ca9ad594562630205968
I20260805 22:46:16.076602 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33441
--builtin_ntp_servers=127.23.197.84:45707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 22:46:16.080458 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.074376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:54331 (local address 127.23.197.126:33441)
0805 22:46:16.074450 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.074452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.074707 (+ 255us) server_negotiation.cc:407] Connection header received
0805 22:46:16.075420 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.075421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.075431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.075495 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:16.076520 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.077241 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.078727 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.079075 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.079168 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.080092 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:16.080094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:16.080095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:16.080100 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:16.080118 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:16.080124 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:16.080189 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.080280 (+ 91us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.080310 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 22:46:16.080575 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.074370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33441 (local address 127.23.197.65:54331)
0805 22:46:16.074568 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:16.074610 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:16.075287 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:16.075575 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:16.075580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:16.075623 (+ 43us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:16.076417 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.076425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.077348 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.077350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.078625 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.078628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.079229 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.079236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.079352 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.079942 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:46:16.079942 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:16.079972 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:46:16.079994 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:16.080191 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:16.080195 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:16.080198 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:16.080252 (+ 54us) client_negotiation.cc:240] Negotiation successful
0805 22:46:16.080331 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260805 22:46:16.081172 19857 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33441
I20260805 22:46:16.081315 19857 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:16.081557 19857 heartbeater.cc:511] Master 127.23.197.126:33441 requested a full tablet report, sending...
I20260805 22:46:16.082088 19578 ts_manager.cc:194] Registered new tserver with Master: b5da4abeda8e427b97aedf205c3636a1 (127.23.197.65:32807)
I20260805 22:46:16.082700 19578 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.65:54331
W20260805 22:46:16.165025 19861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:16.165225 19861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:16.165247 19861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:16.166399 19861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:16.166441 19861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:16.166466 19861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:46:16.167851 19861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:16.168110 19861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:16.168419 19861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:16.173447 19872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:16.173462 19869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:16.173447 19875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:16.173906 19861 server_base.cc:1034] running on GCE node
I20260805 22:46:16.174106 19861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:16.174332 19861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:16.175496 19861 hybrid_clock.cc:648] HybridClock initialized: now 1785969976175487 us; error 40 us; skew 500 ppm
I20260805 22:46:16.176744 19861 webserver.cc:460] Webserver started at http://127.23.197.66:32995/ using document root <none> and password file <none>
I20260805 22:46:16.177069 19861 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:16.177140 19861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:16.177317 19861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:16.178443 19861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "ff945ec939f04f90a1d827a5c3cf9091"
format_stamp: "Formatted at 2026-08-05 22:46:16 on dist-test-slave-g170"
server_key: "3a64d7fed2cc9a3bf321fb4a575d3dde"
server_key_iv: "c241ea81c987152e3fb703bdee2d3362"
server_key_version: "encryptionkey@0"
I20260805 22:46:16.178823 19861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "ff945ec939f04f90a1d827a5c3cf9091"
format_stamp: "Formatted at 2026-08-05 22:46:16 on dist-test-slave-g170"
server_key: "3a64d7fed2cc9a3bf321fb4a575d3dde"
server_key_iv: "c241ea81c987152e3fb703bdee2d3362"
server_key_version: "encryptionkey@0"
I20260805 22:46:16.180199 19861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:16.181030 19884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:16.181236 19861 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:16.181347 19861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "ff945ec939f04f90a1d827a5c3cf9091"
format_stamp: "Formatted at 2026-08-05 22:46:16 on dist-test-slave-g170"
server_key: "3a64d7fed2cc9a3bf321fb4a575d3dde"
server_key_iv: "c241ea81c987152e3fb703bdee2d3362"
server_key_version: "encryptionkey@0"
I20260805 22:46:16.181422 19861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:16.194166 19861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:16.194458 19861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:16.194607 19861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:16.194821 19861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:16.195155 19861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:16.195225 19861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:16.195269 19861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:16.195292 19861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:16.201472 19861 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:41081
I20260805 22:46:16.201530 20053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:41081 every 8 connection(s)
I20260805 22:46:16.201884 19861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:46:16.203351 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 19861
I20260805 22:46:16.203454 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:46:16.203675 24341 external_mini_cluster.cc:1428] Setting key 104efdd4f8e6b011d90bd1607d7717f4
I20260805 22:46:16.204808 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33441
--builtin_ntp_servers=127.23.197.84:45707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 22:46:16.207800 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.202679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:39475 (local address 127.23.197.126:33441)
0805 22:46:16.202773 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.202775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.202990 (+ 215us) server_negotiation.cc:407] Connection header received
0805 22:46:16.203529 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.203531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.203544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.203576 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:16.204623 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.205290 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.206274 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.206489 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.206578 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.207483 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:16.207485 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:16.207485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:16.207490 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:16.207505 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:16.207509 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:16.207585 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.207687 (+ 102us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.207720 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 22:46:16.208063 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.202669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33441 (local address 127.23.197.66:39475)
0805 22:46:16.202858 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:16.202898 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:16.203379 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:16.203758 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:16.203763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:16.203773 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:16.204516 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.204523 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.205388 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.205390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.206173 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.206175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.206588 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.206590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.206744 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.207339 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:46:16.207339 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:16.207367 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:46:16.207389 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:16.207590 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:16.207596 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:16.207600 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:16.207688 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 22:46:16.207791 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260805 22:46:16.208532 20054 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33441
I20260805 22:46:16.208621 20054 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:16.208864 20054 heartbeater.cc:511] Master 127.23.197.126:33441 requested a full tablet report, sending...
I20260805 22:46:16.209251 19578 ts_manager.cc:194] Registered new tserver with Master: ff945ec939f04f90a1d827a5c3cf9091 (127.23.197.66:41081)
I20260805 22:46:16.209583 19578 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.66:39475
W20260805 22:46:16.292594 20059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:16.292856 20059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:16.292943 20059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:16.294806 20059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:16.294855 20059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:16.294891 20059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:46:16.296710 20059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:16.296962 20059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:16.297297 20059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:16.303380 20067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:16.303390 20071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:16.303515 20059 server_base.cc:1034] running on GCE node
W20260805 22:46:16.303443 20068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:16.303866 20059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:16.304165 20059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:16.305384 20059 hybrid_clock.cc:648] HybridClock initialized: now 1785969976305369 us; error 41 us; skew 500 ppm
I20260805 22:46:16.306835 20059 webserver.cc:460] Webserver started at http://127.23.197.67:36681/ using document root <none> and password file <none>
I20260805 22:46:16.307097 20059 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:16.307173 20059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:16.307340 20059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:16.308470 20059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "0ea14d6b32094850b0c0f1cf731183f5"
format_stamp: "Formatted at 2026-08-05 22:46:16 on dist-test-slave-g170"
server_key: "9746be1eec5204929b4da4edf2eb0ca8"
server_key_iv: "86f31269cfffc859f039a6d1d6be613c"
server_key_version: "encryptionkey@0"
I20260805 22:46:16.308933 20059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "0ea14d6b32094850b0c0f1cf731183f5"
format_stamp: "Formatted at 2026-08-05 22:46:16 on dist-test-slave-g170"
server_key: "9746be1eec5204929b4da4edf2eb0ca8"
server_key_iv: "86f31269cfffc859f039a6d1d6be613c"
server_key_version: "encryptionkey@0"
I20260805 22:46:16.310379 20059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:16.311244 20081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:16.311466 20059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:16.311574 20059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "0ea14d6b32094850b0c0f1cf731183f5"
format_stamp: "Formatted at 2026-08-05 22:46:16 on dist-test-slave-g170"
server_key: "9746be1eec5204929b4da4edf2eb0ca8"
server_key_iv: "86f31269cfffc859f039a6d1d6be613c"
server_key_version: "encryptionkey@0"
I20260805 22:46:16.311663 20059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:16.323540 20059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:16.323835 20059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:16.323987 20059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:16.324213 20059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:16.324540 20059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:16.324597 20059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:16.324631 20059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:16.324658 20059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:16.331019 20059 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:44473
I20260805 22:46:16.331063 20274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:44473 every 8 connection(s)
I20260805 22:46:16.331382 20059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:46:16.336549 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.332089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:51427 (local address 127.23.197.126:33441)
0805 22:46:16.332172 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.332174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.332405 (+ 231us) server_negotiation.cc:407] Connection header received
0805 22:46:16.333042 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.333044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.333055 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.333124 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:16.333803 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.334282 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.335246 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.335468 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.335537 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.336306 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:16.336307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:16.336307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:16.336312 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:16.336326 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:16.336330 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:16.336379 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.336464 (+ 85us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.336488 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 22:46:16.336731 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.332069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33441 (local address 127.23.197.67:51427)
0805 22:46:16.332263 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:16.332305 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:16.332881 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:16.333200 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:16.333204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:16.333212 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:16.333720 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.333724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.334374 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.334375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.335143 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.335146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.335556 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.335557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.335663 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.336156 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:46:16.336157 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:16.336184 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:46:16.336206 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:16.336391 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:16.336396 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:16.336398 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:46:16.336449 (+ 51us) client_negotiation.cc:240] Negotiation successful
0805 22:46:16.336524 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260805 22:46:16.337281 20275 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33441
I20260805 22:46:16.337384 20275 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:16.337554 20275 heartbeater.cc:511] Master 127.23.197.126:33441 requested a full tablet report, sending...
I20260805 22:46:16.338002 19578 ts_manager.cc:194] Registered new tserver with Master: 0ea14d6b32094850b0c0f1cf731183f5 (127.23.197.67:44473)
I20260805 22:46:16.338445 19578 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.67:51427
I20260805 22:46:16.341094 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 20059
I20260805 22:46:16.341236 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:46:16.341477 24341 external_mini_cluster.cc:1428] Setting key bd6c9434c6782eb8b1678ec7d8c12682
I20260805 22:46:16.342473 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:46:16.347777 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.344022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.23.197.126:33441)
0805 22:46:16.344089 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.344090 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.344139 (+ 49us) server_negotiation.cc:407] Connection header received
0805 22:46:16.344194 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.344194 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.344202 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.344245 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:16.344412 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.344940 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.346317 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.346625 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.346685 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.347457 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:16.347459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:16.347460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:16.347464 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:16.347477 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:16.347480 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:16.347557 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.347627 (+ 70us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.347636 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 22:46:16.349498 19578 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:46:16.351004 19578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:46:16.357733 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.353460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:32807 (local address 127.0.0.1:45036)
0805 22:46:16.353666 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:16.353708 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:16.353780 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:16.354237 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:16.354238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:16.354241 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:16.354296 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.354298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.355310 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.355312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.356413 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.356416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.357391 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.357392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.357476 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.357491 (+ 15us) client_negotiation.cc:769] Sending connection context
0805 22:46:16.357577 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 22:46:16.357636 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":105,"threads_started":1}
I20260805 22:46:16.357735 20291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.353680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.23.197.65:32807)
0805 22:46:16.354009 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.354015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.354026 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:46:16.354098 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.354100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.354118 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.354381 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:16.354455 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.355207 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.356516 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.357297 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.357372 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.357461 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.357605 (+ 144us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.357649 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":112,"threads_started":1}
I20260805 22:46:16.358081 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.353460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:41081 (local address 127.0.0.1:40816)
0805 22:46:16.353893 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:16.353902 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:16.353942 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:16.354203 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:16.354206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:16.354208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:16.354284 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.354286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.355816 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.355817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.356900 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.356903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.357858 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.357859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.357927 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.357932 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:46:16.357968 (+ 36us) client_negotiation.cc:240] Negotiation successful
0805 22:46:16.358012 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":4,"thread_start_us":80,"threads_started":1}
I20260805 22:46:16.358278 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.353475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:44473 (local address 127.0.0.1:49924)
0805 22:46:16.354109 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:16.354121 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:16.354191 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:16.354512 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:16.354513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:16.354515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:16.354549 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.354550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.355911 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.355913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.356956 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.356959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.358013 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.358014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.358078 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.358082 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:16.358133 (+ 51us) client_negotiation.cc:240] Negotiation successful
0805 22:46:16.358191 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":120,"threads_started":1}
I20260805 22:46:16.358510 20292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.353749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.23.197.66:41081)
0805 22:46:16.353956 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.353962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.353977 (+ 15us) server_negotiation.cc:407] Connection header received
0805 22:46:16.354036 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.354038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.354057 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.354375 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:16.354444 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.355712 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.356994 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.357751 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.358253 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.358364 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.358391 (+ 27us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.358431 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
I20260805 22:46:16.358645 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.353903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49924 (local address 127.23.197.67:44473)
0805 22:46:16.354101 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.354105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.354308 (+ 203us) server_negotiation.cc:407] Connection header received
0805 22:46:16.354355 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.354357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.354373 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.354446 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:16.354616 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.355834 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.357066 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.357790 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.357864 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.357976 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.358468 (+ 492us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.358513 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260805 22:46:16.358594 19755 tablet_service.cc:1467] Processing CreateTablet for tablet d753e4b5552546feacd58a11b74ef525 (DEFAULT_TABLE table=test-table [id=d82abffcbfd1466eb59711e9d4085e5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:16.358999 19755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d753e4b5552546feacd58a11b74ef525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:16.359117 19951 tablet_service.cc:1467] Processing CreateTablet for tablet d753e4b5552546feacd58a11b74ef525 (DEFAULT_TABLE table=test-table [id=d82abffcbfd1466eb59711e9d4085e5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:16.359445 19951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d753e4b5552546feacd58a11b74ef525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:16.362792 20297 tablet_bootstrap.cc:492] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1: Bootstrap starting.
I20260805 22:46:16.362812 20298 tablet_bootstrap.cc:492] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: Bootstrap starting.
I20260805 22:46:16.363148 20165 tablet_service.cc:1467] Processing CreateTablet for tablet d753e4b5552546feacd58a11b74ef525 (DEFAULT_TABLE table=test-table [id=d82abffcbfd1466eb59711e9d4085e5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:16.363481 20165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d753e4b5552546feacd58a11b74ef525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:16.363554 20298 tablet_bootstrap.cc:654] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:16.363554 20297 tablet_bootstrap.cc:654] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:16.363909 20297 log.cc:824] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:16.363909 20298 log.cc:824] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:16.364722 20298 tablet_bootstrap.cc:492] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: No bootstrap required, opened a new log
I20260805 22:46:16.364835 20298 ts_tablet_manager.cc:1397] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:46:16.365084 20297 tablet_bootstrap.cc:492] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1: No bootstrap required, opened a new log
I20260805 22:46:16.365154 20297 ts_tablet_manager.cc:1397] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260805 22:46:16.366626 20302 tablet_bootstrap.cc:492] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: Bootstrap starting.
I20260805 22:46:16.367278 20302 tablet_bootstrap.cc:654] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:16.367585 20302 log.cc:824] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:16.367604 20298 raft_consensus.cc:348] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } }
I20260805 22:46:16.367784 20298 raft_consensus.cc:374] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:16.367832 20298 raft_consensus.cc:729] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff945ec939f04f90a1d827a5c3cf9091, State: Initialized, Role: FOLLOWER
I20260805 22:46:16.367950 20298 consensus_queue.cc:260] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } }
I20260805 22:46:16.368091 20297 raft_consensus.cc:348] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } }
I20260805 22:46:16.368256 20298 ts_tablet_manager.cc:1428] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:46:16.368255 20297 raft_consensus.cc:374] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:16.368295 20297 raft_consensus.cc:729] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5da4abeda8e427b97aedf205c3636a1, State: Initialized, Role: FOLLOWER
I20260805 22:46:16.368371 20054 heartbeater.cc:503] Master 127.23.197.126:33441 was elected leader, sending a full tablet report...
I20260805 22:46:16.368491 20302 tablet_bootstrap.cc:492] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: No bootstrap required, opened a new log
I20260805 22:46:16.368583 20302 ts_tablet_manager.cc:1397] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:46:16.368716 20297 consensus_queue.cc:260] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } }
I20260805 22:46:16.369030 20297 ts_tablet_manager.cc:1428] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260805 22:46:16.369068 19857 heartbeater.cc:503] Master 127.23.197.126:33441 was elected leader, sending a full tablet report...
I20260805 22:46:16.370975 20302 raft_consensus.cc:348] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } }
I20260805 22:46:16.371125 20302 raft_consensus.cc:374] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:16.371177 20302 raft_consensus.cc:729] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ea14d6b32094850b0c0f1cf731183f5, State: Initialized, Role: FOLLOWER
I20260805 22:46:16.371289 20302 consensus_queue.cc:260] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } }
I20260805 22:46:16.371557 20302 ts_tablet_manager.cc:1428] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:16.371626 20275 heartbeater.cc:503] Master 127.23.197.126:33441 was elected leader, sending a full tablet report...
I20260805 22:46:16.391161 20306 raft_consensus.cc:482] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:46:16.391410 20306 raft_consensus.cc:504] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } }
I20260805 22:46:16.391909 20306 leader_election.cc:288] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff945ec939f04f90a1d827a5c3cf9091 (127.23.197.66:41081), 0ea14d6b32094850b0c0f1cf731183f5 (127.23.197.67:44473)
I20260805 22:46:16.395572 20292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.392203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:55793 (local address 127.23.197.66:41081)
0805 22:46:16.392290 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.392298 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.392311 (+ 13us) server_negotiation.cc:407] Connection header received
0805 22:46:16.392361 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.392362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.392373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.392439 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:16.392705 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.393496 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.394863 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.395225 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.395285 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.395300 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.395487 (+ 187us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.395517 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:46:16.395598 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.392073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:41081 (local address 127.23.197.65:55793)
0805 22:46:16.392209 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:16.392223 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:16.392275 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:16.392448 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:16.392449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:16.392483 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:16.392589 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.392592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.393590 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.393592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.394765 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.394767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.395319 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.395320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.395404 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.395409 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 22:46:16.395474 (+ 65us) client_negotiation.cc:240] Negotiation successful
0805 22:46:16.395517 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260805 22:46:16.395952 19993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d753e4b5552546feacd58a11b74ef525" candidate_uuid: "b5da4abeda8e427b97aedf205c3636a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff945ec939f04f90a1d827a5c3cf9091" is_pre_election: true
I20260805 22:46:16.396122 19993 raft_consensus.cc:2454] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5da4abeda8e427b97aedf205c3636a1 in term 0.
I20260805 22:46:16.396377 19692 leader_election.cc:302] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5da4abeda8e427b97aedf205c3636a1, ff945ec939f04f90a1d827a5c3cf9091; no voters:
I20260805 22:46:16.396582 20306 raft_consensus.cc:2781] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:46:16.396672 20306 raft_consensus.cc:482] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:46:16.396714 20306 raft_consensus.cc:3037] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:16.396763 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.393478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:50051 (local address 127.23.197.67:44473)
0805 22:46:16.393709 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.393711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.393721 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:46:16.393753 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.393754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.393765 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.393816 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:16.394033 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.394495 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.395771 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.396359 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.396450 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.396459 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.396616 (+ 157us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.396687 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260805 22:46:16.396831 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.392483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:44473 (local address 127.23.197.65:50051)
0805 22:46:16.392699 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:16.392711 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:16.392825 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:16.393843 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:16.393844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:16.393847 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:16.393915 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.393916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.394554 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.394556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.395687 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:16.395690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.396450 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.396451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:16.396531 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.396535 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:16.396595 (+ 60us) client_negotiation.cc:240] Negotiation successful
0805 22:46:16.396639 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":128,"threads_started":1}
I20260805 22:46:16.397262 20202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d753e4b5552546feacd58a11b74ef525" candidate_uuid: "b5da4abeda8e427b97aedf205c3636a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ea14d6b32094850b0c0f1cf731183f5" is_pre_election: true
I20260805 22:46:16.397392 20202 raft_consensus.cc:2454] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5da4abeda8e427b97aedf205c3636a1 in term 0.
I20260805 22:46:16.397799 20306 raft_consensus.cc:504] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } }
I20260805 22:46:16.397957 20306 leader_election.cc:288] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [CANDIDATE]: Term 1 election: Requested vote from peers ff945ec939f04f90a1d827a5c3cf9091 (127.23.197.66:41081), 0ea14d6b32094850b0c0f1cf731183f5 (127.23.197.67:44473)
I20260805 22:46:16.398118 20202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d753e4b5552546feacd58a11b74ef525" candidate_uuid: "b5da4abeda8e427b97aedf205c3636a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ea14d6b32094850b0c0f1cf731183f5"
I20260805 22:46:16.398172 19993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d753e4b5552546feacd58a11b74ef525" candidate_uuid: "b5da4abeda8e427b97aedf205c3636a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff945ec939f04f90a1d827a5c3cf9091"
I20260805 22:46:16.398234 20202 raft_consensus.cc:3037] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:16.398272 19993 raft_consensus.cc:3037] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:16.399118 19993 raft_consensus.cc:2454] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5da4abeda8e427b97aedf205c3636a1 in term 1.
I20260805 22:46:16.399178 20202 raft_consensus.cc:2454] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5da4abeda8e427b97aedf205c3636a1 in term 1.
I20260805 22:46:16.399374 19692 leader_election.cc:302] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5da4abeda8e427b97aedf205c3636a1, ff945ec939f04f90a1d827a5c3cf9091; no voters:
I20260805 22:46:16.399482 20306 raft_consensus.cc:2781] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:16.399663 20306 raft_consensus.cc:686] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 1 LEADER]: Becoming Leader. State: Replica: b5da4abeda8e427b97aedf205c3636a1, State: Running, Role: LEADER
I20260805 22:46:16.399784 20306 consensus_queue.cc:237] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } }
I20260805 22:46:16.400662 19578 catalog_manager.cc:5495] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5da4abeda8e427b97aedf205c3636a1 (127.23.197.65). New cstate: current_term: 1 leader_uuid: "b5da4abeda8e427b97aedf205c3636a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5da4abeda8e427b97aedf205c3636a1" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 32807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 } health_report { overall_health: UNKNOWN } } }
I20260805 22:46:16.417428 20291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.414669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45050 (local address 127.23.197.65:32807)
0805 22:46:16.414738 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:16.414740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:16.414765 (+ 25us) server_negotiation.cc:407] Connection header received
0805 22:46:16.414800 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:16.414801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:16.414822 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:16.414881 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:16.415016 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.415523 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.416435 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:16.416715 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:16.416890 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:16.417042 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:16.417140 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:16.417249 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:16.417334 (+ 85us) server_negotiation.cc:299] Negotiation successful
0805 22:46:16.417346 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 22:46:16.419590 20202 raft_consensus.cc:1261] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [term 1 FOLLOWER]: Refusing update from remote peer b5da4abeda8e427b97aedf205c3636a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:16.419598 19993 raft_consensus.cc:1261] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [term 1 FOLLOWER]: Refusing update from remote peer b5da4abeda8e427b97aedf205c3636a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:16.419981 20310 consensus_queue.cc:1035] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ea14d6b32094850b0c0f1cf731183f5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 44473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:16.420145 20306 consensus_queue.cc:1035] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff945ec939f04f90a1d827a5c3cf9091" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:16.423270 20321 mvcc.cc:205] Tried to move back new op lower bound from 7315333023411224576 to 7315333023334277120. Current Snapshot: MvccSnapshot[applied={T|T < 7315333023411224576}]
I20260805 22:46:16.425377 19571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:16.425482 19571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:16.426263 19571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 386fb95c0a904372a3799fa2ecadea09: Sending DeleteTablet for 3 replicas of tablet d753e4b5552546feacd58a11b74ef525
I20260805 22:46:16.426581 19951 tablet_service.cc:1514] Processing DeleteTablet for tablet d753e4b5552546feacd58a11b74ef525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:16 UTC) from {username='slave'} at 127.0.0.1:40816
I20260805 22:46:16.426594 19755 tablet_service.cc:1514] Processing DeleteTablet for tablet d753e4b5552546feacd58a11b74ef525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:16 UTC) from {username='slave'} at 127.0.0.1:45036
I20260805 22:46:16.426854 20165 tablet_service.cc:1514] Processing DeleteTablet for tablet d753e4b5552546feacd58a11b74ef525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:16 UTC) from {username='slave'} at 127.0.0.1:49924
I20260805 22:46:16.426909 20336 tablet_replica.cc:330] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1: stopping tablet replica
I20260805 22:46:16.427064 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 19671
I20260805 22:46:16.427101 20336 raft_consensus.cc:2229] T d753e4b5552546feacd58a11b74ef525 P b5da4abeda8e427b97aedf205c3636a1 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:46:16.429700 20335 tablet_replica.cc:330] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: stopping tablet replica
I20260805 22:46:16.429819 20335 raft_consensus.cc:2229] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:46:16.429932 20335 raft_consensus.cc:2258] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:16.431011 20335 ts_tablet_manager.cc:1905] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:16.433313 20335 ts_tablet_manager.cc:1918] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:16.433388 20335 log.cc:1196] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/d753e4b5552546feacd58a11b74ef525
I20260805 22:46:16.433616 20335 ts_tablet_manager.cc:1939] T d753e4b5552546feacd58a11b74ef525 P ff945ec939f04f90a1d827a5c3cf9091: Deleting consensus metadata
I20260805 22:46:16.435169 19551 catalog_manager.cc:4841] TS ff945ec939f04f90a1d827a5c3cf9091 (127.23.197.66:41081): tablet d753e4b5552546feacd58a11b74ef525 (table test-table [id=d82abffcbfd1466eb59711e9d4085e5d]) successfully deleted
I20260805 22:46:16.436225 20338 tablet_replica.cc:330] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: stopping tablet replica
I20260805 22:46:16.436354 20338 raft_consensus.cc:2229] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:46:16.436468 20338 raft_consensus.cc:2258] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:16.436939 20338 ts_tablet_manager.cc:1905] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:16.438974 20338 ts_tablet_manager.cc:1918] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:16.439079 20338 log.cc:1196] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/d753e4b5552546feacd58a11b74ef525
I20260805 22:46:16.439378 20338 ts_tablet_manager.cc:1939] T d753e4b5552546feacd58a11b74ef525 P 0ea14d6b32094850b0c0f1cf731183f5: Deleting consensus metadata
I20260805 22:46:16.439893 19552 catalog_manager.cc:4841] TS 0ea14d6b32094850b0c0f1cf731183f5 (127.23.197.67:44473): tablet d753e4b5552546feacd58a11b74ef525 (table test-table [id=d82abffcbfd1466eb59711e9d4085e5d]) successfully deleted
W20260805 22:46:16.441654 19550 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:32807 (error 108)
I20260805 22:46:16.441833 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 19861
I20260805 22:46:16.444139 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:16.443965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:32807 (local address 127.0.0.1:45064)
0805 22:46:16.444030 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:16.444093 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260805 22:46:16.445312 19550 catalog_manager.cc:4584] TS b5da4abeda8e427b97aedf205c3636a1 (127.23.197.65:32807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d753e4b5552546feacd58a11b74ef525: Network error: Client connection negotiation failed: client connection to 127.23.197.65:32807: connect: Connection refused (error 111)
I20260805 22:46:16.450754 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 20059
I20260805 22:46:16.472072 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 19519
2026-08-05T22:46:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (716 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:16 dist-test-slave-g170 krb5kdc[20342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:16 dist-test-slave-g170 krb5kdc[20342](info): set up 2 sockets
Aug 05 22:46:16 dist-test-slave-g170 krb5kdc[20342](info): commencing operation
krb5kdc: starting...
W20260805 22:46:17.712088 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:17 dist-test-slave-g170 krb5kdc[20342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:17.841954 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33765
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:33765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:17.927836 20364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:17.928110 20364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:17.928151 20364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:17.929440 20364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:17.929529 20364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:17.929555 20364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:17.929577 20364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:17.929634 20364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:17.931269 20364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:33765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:17.931571 20364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:17.931825 20364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:17.937189 20378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:17.937214 20380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:17.937193 20377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:17.937572 20364 server_base.cc:1034] running on GCE node
I20260805 22:46:17.937878 20364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:17.938259 20364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:17.939428 20364 hybrid_clock.cc:648] HybridClock initialized: now 1785969977939394 us; error 52 us; skew 500 ppm
Aug 05 22:46:17 dist-test-slave-g170 krb5kdc[20342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969977, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:17.946043 20364 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:17.946636 20364 webserver.cc:460] Webserver started at http://127.23.197.126:43539/ using document root <none> and password file <none>
I20260805 22:46:17.946904 20364 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:17.946983 20364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:17.947158 20364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:17.948454 20364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45"
format_stamp: "Formatted at 2026-08-05 22:46:17 on dist-test-slave-g170"
server_key: "c4318760a98b14f2b36f0cc5e6135c9e"
server_key_iv: "8ba527f564f63b38e745633c73b7432d"
server_key_version: "encryptionkey@0"
I20260805 22:46:17.948974 20364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45"
format_stamp: "Formatted at 2026-08-05 22:46:17 on dist-test-slave-g170"
server_key: "c4318760a98b14f2b36f0cc5e6135c9e"
server_key_iv: "8ba527f564f63b38e745633c73b7432d"
server_key_version: "encryptionkey@0"
I20260805 22:46:17.950727 20364 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260805 22:46:17.951701 20391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:17.951977 20364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:17.952117 20364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45"
format_stamp: "Formatted at 2026-08-05 22:46:17 on dist-test-slave-g170"
server_key: "c4318760a98b14f2b36f0cc5e6135c9e"
server_key_iv: "8ba527f564f63b38e745633c73b7432d"
server_key_version: "encryptionkey@0"
I20260805 22:46:17.952222 20364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:17.968953 20364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:17.971470 20364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:17.971683 20364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:17.976950 20364 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:33765
I20260805 22:46:17.977058 20463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:33765 every 8 connection(s)
I20260805 22:46:17.977489 20364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:17.978214 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 20364
I20260805 22:46:17.978363 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:17.978583 24341 external_mini_cluster.cc:1428] Setting key ee1bad4a83a13ed8994526efcc3976b4
I20260805 22:46:17.978663 20464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:17.981967 20464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45: Bootstrap starting.
I20260805 22:46:17.982671 20464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:17.983007 20464 log.cc:824] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:17.983881 20464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45: No bootstrap required, opened a new log
Aug 05 22:46:17 dist-test-slave-g170 krb5kdc[20342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:17.986745 20464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33765 } }
I20260805 22:46:17.986932 20464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:17.986990 20464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f86bfe66a1b4ddfaea32c1c4d072f45, State: Initialized, Role: FOLLOWER
I20260805 22:46:17.987161 20464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33765 } }
I20260805 22:46:17.987247 20464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:17.987273 20464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:17.987346 20464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:17.987965 20464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33765 } }
I20260805 22:46:17.988101 20464 leader_election.cc:302] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f86bfe66a1b4ddfaea32c1c4d072f45; no voters:
I20260805 22:46:17.988350 20464 leader_election.cc:288] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:17.988389 20471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:17.988634 20471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [term 1 LEADER]: Becoming Leader. State: Replica: 7f86bfe66a1b4ddfaea32c1c4d072f45, State: Running, Role: LEADER
I20260805 22:46:17.988701 20464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:17.988744 20471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33765 } }
I20260805 22:46:17.989156 20471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f86bfe66a1b4ddfaea32c1c4d072f45. Latest consensus state: current_term: 1 leader_uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33765 } } }
I20260805 22:46:17.989243 20471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:17.990226 20472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f86bfe66a1b4ddfaea32c1c4d072f45" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33765 } } }
I20260805 22:46:17.990331 20472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:17.990706 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:17.979179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.23.197.126:33765)
0805 22:46:17.979407 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:17.979416 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:17.979442 (+ 26us) server_negotiation.cc:407] Connection header received
0805 22:46:17.979875 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:17.979881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:17.979930 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:17.980088 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:17.980753 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:17.981684 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:17.982549 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:17.983041 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:17.983178 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:17.986362 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:17.986366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:17.986367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:17.986394 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:17.988348 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:17.988901 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:17.988906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:17.988906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:17.989012 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:17.989258 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:17.989259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:17.989260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:17.989753 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:17.990235 (+ 482us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:17.990329 (+ 94us) server_negotiation.cc:299] Negotiation successful
0805 22:46:17.990509 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
W20260805 22:46:17.991842 20492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:17.991909 20492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:17.992910 20495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:17.993167 20495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:17.995024 20495 catalog_manager.cc:1347] Generated new cluster ID: f38b07acbaa44e5ba104c2aec8d7ac30
I20260805 22:46:17.995097 20495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:18.012570 20495 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:18.013198 20495 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:18.022174 20495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45: Generated new TSK 0
I20260805 22:46:18.022409 20495 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.197.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
Entry for principal kudu/127.23.197.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab.
I20260805 22:46:18.059531 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33765
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:18.155849 20506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:18.156056 20506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:18.156080 20506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:18.157369 20506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:18.157413 20506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:18.157440 20506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:46:18.158927 20506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.65.keytab
--principal=kudu/127.23.197.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:18.159185 20506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:18.159426 20506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:18.164508 20518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.164516 20517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.164531 20522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:18.164788 20506 server_base.cc:1034] running on GCE node
I20260805 22:46:18.165071 20506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:18.165346 20506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:18.166543 20506 hybrid_clock.cc:648] HybridClock initialized: now 1785969978166531 us; error 30 us; skew 500 ppm
Aug 05 22:46:18 dist-test-slave-g170 krb5kdc[20342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969978, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:18.169318 20506 init.cc:377] Logged in from keytab as kudu/127.23.197.65@KRBTEST.COM (short username kudu)
I20260805 22:46:18.169826 20506 webserver.cc:460] Webserver started at http://127.23.197.65:36647/ using document root <none> and password file <none>
I20260805 22:46:18.170076 20506 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:18.170146 20506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:18.170296 20506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:18.171393 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "c539dce7388644b297567440a89aa79a"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "2f51649fb6f061c2eba2252c6d3f6602"
server_key_iv: "e58e92e42ac2b195c69fc2f0344b47e5"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.171777 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "c539dce7388644b297567440a89aa79a"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "2f51649fb6f061c2eba2252c6d3f6602"
server_key_iv: "e58e92e42ac2b195c69fc2f0344b47e5"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.173202 20506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:18.174039 20536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.174239 20506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:18.174347 20506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "c539dce7388644b297567440a89aa79a"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "2f51649fb6f061c2eba2252c6d3f6602"
server_key_iv: "e58e92e42ac2b195c69fc2f0344b47e5"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.174422 20506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:18.185815 20506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:18.187816 20506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:18.187990 20506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:18.188228 20506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:18.188557 20506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:18.188664 20506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.188727 20506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:18.188757 20506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.195158 20506 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:43775
I20260805 22:46:18.195226 20696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:43775 every 8 connection(s)
I20260805 22:46:18.195538 20506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:46:18.195983 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 20506
I20260805 22:46:18.196112 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:46:18.196399 24341 external_mini_cluster.cc:1428] Setting key 057b4eb59cda4be8c1880f0647154c28
Aug 05 22:46:18 dist-test-slave-g170 krb5kdc[20342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969978, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.65@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.197.66@KRBTEST.COM; defaulting to no policy
I20260805 22:46:18.206820 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.196296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:44201 (local address 127.23.197.126:33765)
0805 22:46:18.196351 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.196354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.196613 (+ 259us) server_negotiation.cc:407] Connection header received
0805 22:46:18.197332 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.197333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.197342 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.197373 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:18.198751 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.199277 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.200830 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.201065 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.201132 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.203769 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:18.203770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:18.203771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:18.203779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:18.205330 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:18.205769 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:18.205771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:18.205772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:18.205841 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:18.206112 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:18.206113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:18.206113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:18.206232 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:18.206314 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.206610 (+ 296us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.206719 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:46:18.206982 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.196296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33765 (local address 127.23.197.65:44201)
0805 22:46:18.196480 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:18.196533 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:18.197224 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:18.197462 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:18.197466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:18.198345 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:18.198620 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.198626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.199363 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.199364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.200665 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.200679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.201164 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.201165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.201337 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.201891 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:18.201902 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:18.203665 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:18.205437 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:18.205446 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:18.205448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:18.205685 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:18.205918 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:18.205919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:18.205919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:18.206035 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:18.206309 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:18.206317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:18.206467 (+ 150us) client_negotiation.cc:769] Sending connection context
0805 22:46:18.206574 (+ 107us) client_negotiation.cc:240] Negotiation successful
0805 22:46:18.206734 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260805 22:46:18.207590 20697 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33765
I20260805 22:46:18.207705 20697 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:18.207896 20697 heartbeater.cc:511] Master 127.23.197.126:33765 requested a full tablet report, sending...
I20260805 22:46:18.208527 20412 ts_manager.cc:194] Registered new tserver with Master: c539dce7388644b297567440a89aa79a (127.23.197.65:43775)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.66@KRBTEST.COM" created.
I20260805 22:46:18.209306 20412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.65@KRBTEST.COM'} at 127.23.197.65:44201
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
Entry for principal kudu/127.23.197.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab.
I20260805 22:46:18.224076 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33765
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:18.310711 20705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:18.310926 20705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:18.310950 20705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:18.312134 20705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:18.312183 20705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:18.312212 20705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:46:18.313763 20705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.66.keytab
--principal=kudu/127.23.197.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:18.314031 20705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:18.314283 20705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:18.319478 20713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.319491 20714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.319517 20716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:18.319921 20705 server_base.cc:1034] running on GCE node
I20260805 22:46:18.320129 20705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:18.320397 20705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:18.321571 20705 hybrid_clock.cc:648] HybridClock initialized: now 1785969978321549 us; error 47 us; skew 500 ppm
Aug 05 22:46:18 dist-test-slave-g170 krb5kdc[20342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969978, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:18.324448 20705 init.cc:377] Logged in from keytab as kudu/127.23.197.66@KRBTEST.COM (short username kudu)
I20260805 22:46:18.324915 20705 webserver.cc:460] Webserver started at http://127.23.197.66:32991/ using document root <none> and password file <none>
I20260805 22:46:18.325124 20705 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:18.325171 20705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:18.325271 20705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:18.326344 20705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "781e922634c341e1a8a36384bd32f9e6"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "5cfa49def10b63cf140d4639f992b785"
server_key_iv: "e885579418bdc9ad0c6efed6624c07ac"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.326651 20705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "781e922634c341e1a8a36384bd32f9e6"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "5cfa49def10b63cf140d4639f992b785"
server_key_iv: "e885579418bdc9ad0c6efed6624c07ac"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.328109 20705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:18.329015 20725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.329233 20705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:18.329325 20705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "781e922634c341e1a8a36384bd32f9e6"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "5cfa49def10b63cf140d4639f992b785"
server_key_iv: "e885579418bdc9ad0c6efed6624c07ac"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.329393 20705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:18.337347 20705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:18.339519 20705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:18.339779 20705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:18.340001 20705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:18.340297 20705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:18.340333 20705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.340358 20705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:18.340372 20705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.347118 20705 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:37005
I20260805 22:46:18.347210 20892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:37005 every 8 connection(s)
I20260805 22:46:18.347561 20705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:46:18.349958 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 20705
I20260805 22:46:18.350062 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:46:18.350293 24341 external_mini_cluster.cc:1428] Setting key 76d063f4db2149e53e276c13d3b89daf
Aug 05 22:46:18 dist-test-slave-g170 krb5kdc[20342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969978, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.66@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:18.359452 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.348399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:38371 (local address 127.23.197.126:33765)
0805 22:46:18.348488 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.348490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.348857 (+ 367us) server_negotiation.cc:407] Connection header received
0805 22:46:18.349356 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.349357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.349369 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.349434 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:18.350707 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.351374 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.353289 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.353622 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.353719 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.356403 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:18.356405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:18.356405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:18.356415 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:18.357978 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:18.358419 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:18.358421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:18.358421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:18.358489 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:18.358767 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:18.358768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:18.358768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:18.358886 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:18.358965 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.359269 (+ 304us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.359359 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 22:46:18.359659 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.348357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33765 (local address 127.23.197.66:38371)
0805 22:46:18.348582 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:18.348670 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:18.349232 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:18.349522 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:18.349525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:18.350384 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:18.350631 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.350637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.351483 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.351485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.353059 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.353062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.353749 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.353750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.353882 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.354505 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:18.354514 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:18.356301 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:18.358084 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:18.358092 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:18.358094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:18.358339 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:18.358564 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:18.358565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:18.358565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:18.358696 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:18.358957 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:18.358963 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:18.359123 (+ 160us) client_negotiation.cc:769] Sending connection context
0805 22:46:18.359236 (+ 113us) client_negotiation.cc:240] Negotiation successful
0805 22:46:18.359400 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.23.197.67@KRBTEST.COM; defaulting to no policy
I20260805 22:46:18.360191 20893 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33765
I20260805 22:46:18.360306 20893 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:18.360522 20893 heartbeater.cc:511] Master 127.23.197.126:33765 requested a full tablet report, sending...
I20260805 22:46:18.361223 20412 ts_manager.cc:194] Registered new tserver with Master: 781e922634c341e1a8a36384bd32f9e6 (127.23.197.66:37005)
I20260805 22:46:18.361584 20412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.66@KRBTEST.COM'} at 127.23.197.66:38371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
Entry for principal kudu/127.23.197.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab.
I20260805 22:46:18.380860 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33765
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:18.470527 20903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:18.470753 20903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:18.470790 20903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:18.471963 20903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:18.472034 20903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:18.472077 20903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:46:18.473557 20903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.67.keytab
--principal=kudu/127.23.197.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:33765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:18.473826 20903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:18.474058 20903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:18.479300 20915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.479370 20913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.479323 20912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:18.479405 20903 server_base.cc:1034] running on GCE node
I20260805 22:46:18.479733 20903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:18.479945 20903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:18.481128 20903 hybrid_clock.cc:648] HybridClock initialized: now 1785969978481092 us; error 60 us; skew 500 ppm
Aug 05 22:46:18 dist-test-slave-g170 krb5kdc[20342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969978, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:18.484033 20903 init.cc:377] Logged in from keytab as kudu/127.23.197.67@KRBTEST.COM (short username kudu)
I20260805 22:46:18.484568 20903 webserver.cc:460] Webserver started at http://127.23.197.67:40303/ using document root <none> and password file <none>
I20260805 22:46:18.484889 20903 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:18.484962 20903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:18.485136 20903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:18.486227 20903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "8025628a656b4de09dab8037d49adb8a"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "2933d9ccf65e0323d903a91f4d11c0e9"
server_key_iv: "ed4dd524c4b47993b304a3b12a51487a"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.486611 20903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "8025628a656b4de09dab8037d49adb8a"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "2933d9ccf65e0323d903a91f4d11c0e9"
server_key_iv: "ed4dd524c4b47993b304a3b12a51487a"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.488080 20903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:18.489063 20928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.489308 20903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:18.489423 20903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "8025628a656b4de09dab8037d49adb8a"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "2933d9ccf65e0323d903a91f4d11c0e9"
server_key_iv: "ed4dd524c4b47993b304a3b12a51487a"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.489522 20903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:18.508316 20903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:18.510541 20903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:18.510711 20903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:18.510959 20903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:18.511288 20903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:18.511344 20903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.511387 20903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:18.511415 20903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.517706 20903 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:41631
I20260805 22:46:18.517758 21102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:41631 every 8 connection(s)
I20260805 22:46:18.518062 20903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
Aug 05 22:46:18 dist-test-slave-g170 krb5kdc[20342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969978, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.67@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:18.527206 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 20903
I20260805 22:46:18.527314 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:46:18.527525 24341 external_mini_cluster.cc:1428] Setting key 0319f3e6dc742909f3298335673beac3
I20260805 22:46:18.528599 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.518888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:48825 (local address 127.23.197.126:33765)
0805 22:46:18.519010 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.519013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.519248 (+ 235us) server_negotiation.cc:407] Connection header received
0805 22:46:18.519844 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.519845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.519860 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.519917 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:18.521051 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.521611 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.522809 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.523047 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.523135 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.525479 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:18.525480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:18.525481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:18.525488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:18.526982 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:18.527513 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:18.527514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:18.527514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:18.527598 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:18.527927 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:18.527928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:18.527928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:18.528051 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:18.528131 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.528425 (+ 294us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.528508 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260805 22:46:18.528972 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.518786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:33765 (local address 127.23.197.67:48825)
0805 22:46:18.519051 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:18.519131 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:18.519711 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:18.520000 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:18.520003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:18.520644 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:18.520936 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.520942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.521738 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.521739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.522711 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.522713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.523144 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.523146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.523281 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.523812 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:18.523820 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:18.525377 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:18.527080 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:18.527087 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:18.527089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:18.527432 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:18.527696 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:18.527697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:18.527698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:18.527869 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:18.528122 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:18.528128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:18.528278 (+ 150us) client_negotiation.cc:769] Sending connection context
0805 22:46:18.528387 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 22:46:18.528548 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":120,"threads_started":1}
I20260805 22:46:18.529489 21103 heartbeater.cc:347] Connected to a master server at 127.23.197.126:33765
I20260805 22:46:18.529573 21103 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:18.529716 21103 heartbeater.cc:511] Master 127.23.197.126:33765 requested a full tablet report, sending...
I20260805 22:46:18.530037 20412 ts_manager.cc:194] Registered new tserver with Master: 8025628a656b4de09dab8037d49adb8a (127.23.197.67:41631)
I20260805 22:46:18.530404 20412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.197.67@KRBTEST.COM'} at 127.23.197.67:48825
I20260805 22:46:18.539453 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:46:18.546862 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.540826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.23.197.126:33765)
0805 22:46:18.540867 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.540869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.540935 (+ 66us) server_negotiation.cc:407] Connection header received
0805 22:46:18.540986 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.540986 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.540994 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.541042 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:18.541590 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.542078 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.542792 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.543017 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.543083 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.544322 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:18.544323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:18.544323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:18.544329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:18.545808 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:18.546172 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:18.546172 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:18.546173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:18.546221 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:18.546404 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:18.546405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:18.546405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:18.546539 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:18.546656 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.546739 (+ 83us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.546800 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:46:18.548753 20412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:46:18.550470 20412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:46:18.557797 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.553122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:41631 (local address 127.0.0.1:45466)
0805 22:46:18.553614 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:18.553624 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:18.553690 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:18.553965 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:18.553968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:18.553971 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:18.554062 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.554063 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.555399 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.555400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.556685 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.556687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.557409 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.557410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.557510 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.557525 (+ 15us) client_negotiation.cc:769] Sending connection context
0805 22:46:18.557639 (+ 114us) client_negotiation.cc:240] Negotiation successful
0805 22:46:18.557701 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":47,"thread_start_us":100,"threads_started":1}
I20260805 22:46:18.557814 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.553229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.23.197.67:41631)
0805 22:46:18.553557 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.553561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.553687 (+ 126us) server_negotiation.cc:407] Connection header received
0805 22:46:18.553784 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.553787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.553803 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.553871 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:18.554160 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.555320 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.556756 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.557310 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.557410 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.557473 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.557661 (+ 188us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.557709 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":113,"threads_started":1}
I20260805 22:46:18.557814 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.553422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.23.197.66:37005)
0805 22:46:18.553592 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.553605 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.553713 (+ 108us) server_negotiation.cc:407] Connection header received
0805 22:46:18.553759 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.553761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.553778 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.553864 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:18.554153 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.555270 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.556555 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.557332 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.557411 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.557471 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.557696 (+ 225us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.557737 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260805 22:46:18.557999 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.553252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52694 (local address 127.23.197.65:43775)
0805 22:46:18.553469 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.553474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.553686 (+ 212us) server_negotiation.cc:407] Connection header received
0805 22:46:18.553845 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.553848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.553868 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.553952 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:18.554249 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.555012 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.556669 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.557359 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.557417 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.557471 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.557848 (+ 377us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.557896 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":87,"threads_started":1}
I20260805 22:46:18.557806 21116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.553107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:43775 (local address 127.0.0.1:52694)
0805 22:46:18.553447 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:18.553493 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:18.553580 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:18.554046 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:18.554047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:18.554049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:18.554153 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.554155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.555122 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.555123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.556579 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.556582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.557511 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.557511 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.557591 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.557608 (+ 17us) client_negotiation.cc:769] Sending connection context
0805 22:46:18.557653 (+ 45us) client_negotiation.cc:240] Negotiation successful
0805 22:46:18.557714 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260805 22:46:18.558200 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.553309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:37005 (local address 127.0.0.1:59686)
0805 22:46:18.553616 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:18.553626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:18.553685 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:18.553965 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:18.553968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:18.553974 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:18.554052 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.554054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.555390 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.555391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.556446 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.556449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.557509 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.557511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.557591 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.557598 (+ 7us) client_negotiation.cc:769] Sending connection context
0805 22:46:18.557653 (+ 55us) client_negotiation.cc:240] Negotiation successful
0805 22:46:18.558120 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":417,"thread_start_us":76,"threads_started":1}
I20260805 22:46:18.558827 20997 tablet_service.cc:1467] Processing CreateTablet for tablet 95f931be7dc6408cb7f880f0f3b7dc28 (DEFAULT_TABLE table=test-table [id=605dd2aedf52416ca214fa18b388b5e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:18.558830 20604 tablet_service.cc:1467] Processing CreateTablet for tablet 95f931be7dc6408cb7f880f0f3b7dc28 (DEFAULT_TABLE table=test-table [id=605dd2aedf52416ca214fa18b388b5e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:18.559242 20604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f931be7dc6408cb7f880f0f3b7dc28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:18.559237 20997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f931be7dc6408cb7f880f0f3b7dc28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:18.561544 20805 tablet_service.cc:1467] Processing CreateTablet for tablet 95f931be7dc6408cb7f880f0f3b7dc28 (DEFAULT_TABLE table=test-table [id=605dd2aedf52416ca214fa18b388b5e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:18.561915 20805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f931be7dc6408cb7f880f0f3b7dc28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:18.563082 21126 tablet_bootstrap.cc:492] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: Bootstrap starting.
I20260805 22:46:18.563915 21126 tablet_bootstrap.cc:654] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:18.564584 21126 log.cc:824] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:18.565030 21125 tablet_bootstrap.cc:492] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a: Bootstrap starting.
I20260805 22:46:18.565474 21130 tablet_bootstrap.cc:492] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: Bootstrap starting.
I20260805 22:46:18.565752 21125 tablet_bootstrap.cc:654] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:18.566148 21130 tablet_bootstrap.cc:654] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:18.566346 21125 log.cc:824] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:18.566768 21130 log.cc:824] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:18.566982 21126 tablet_bootstrap.cc:492] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: No bootstrap required, opened a new log
I20260805 22:46:18.567075 21126 ts_tablet_manager.cc:1397] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260805 22:46:18.567243 21125 tablet_bootstrap.cc:492] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a: No bootstrap required, opened a new log
I20260805 22:46:18.567508 21125 ts_tablet_manager.cc:1397] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 22:46:18.567744 21130 tablet_bootstrap.cc:492] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: No bootstrap required, opened a new log
I20260805 22:46:18.567827 21130 ts_tablet_manager.cc:1397] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:46:18.569175 21126 raft_consensus.cc:348] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } }
I20260805 22:46:18.569352 21126 raft_consensus.cc:374] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:18.569403 21126 raft_consensus.cc:729] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8025628a656b4de09dab8037d49adb8a, State: Initialized, Role: FOLLOWER
I20260805 22:46:18.569558 21126 consensus_queue.cc:260] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } }
I20260805 22:46:18.569603 21125 raft_consensus.cc:348] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } }
I20260805 22:46:18.569792 21125 raft_consensus.cc:374] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:18.569834 21125 raft_consensus.cc:729] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c539dce7388644b297567440a89aa79a, State: Initialized, Role: FOLLOWER
I20260805 22:46:18.569890 21126 ts_tablet_manager.cc:1428] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260805 22:46:18.570025 21103 heartbeater.cc:503] Master 127.23.197.126:33765 was elected leader, sending a full tablet report...
I20260805 22:46:18.569988 21125 consensus_queue.cc:260] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } }
I20260805 22:46:18.570003 21130 raft_consensus.cc:348] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } }
I20260805 22:46:18.570176 21130 raft_consensus.cc:374] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:18.570227 21130 raft_consensus.cc:729] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781e922634c341e1a8a36384bd32f9e6, State: Initialized, Role: FOLLOWER
I20260805 22:46:18.570317 21125 ts_tablet_manager.cc:1428] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260805 22:46:18.570443 20697 heartbeater.cc:503] Master 127.23.197.126:33765 was elected leader, sending a full tablet report...
I20260805 22:46:18.570367 21130 consensus_queue.cc:260] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } }
I20260805 22:46:18.571096 21130 ts_tablet_manager.cc:1428] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:18.571281 20893 heartbeater.cc:503] Master 127.23.197.126:33765 was elected leader, sending a full tablet report...
I20260805 22:46:18.585660 21135 raft_consensus.cc:482] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:46:18.585855 21135 raft_consensus.cc:504] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } }
I20260805 22:46:18.586247 21135 leader_election.cc:288] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c539dce7388644b297567440a89aa79a (127.23.197.65:43775), 8025628a656b4de09dab8037d49adb8a (127.23.197.67:41631)
I20260805 22:46:18.589180 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.586547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:40567 (local address 127.23.197.65:43775)
0805 22:46:18.586589 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.586591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.586600 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:18.586626 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.586626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.586637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.586723 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:18.586874 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.587354 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.588258 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.588732 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.588900 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.588910 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.589076 (+ 166us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.589113 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:46:18.589191 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.586435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:43775 (local address 127.23.197.66:40567)
0805 22:46:18.586498 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:18.586509 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:18.586555 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:18.586734 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:18.586735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:18.586737 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:18.586804 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.586805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.587436 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.587437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.588158 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.588160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.588897 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.588898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.588997 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.589001 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:18.589071 (+ 70us) client_negotiation.cc:240] Negotiation successful
0805 22:46:18.589112 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":4}
I20260805 22:46:18.589339 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.586537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:34855 (local address 127.23.197.67:41631)
0805 22:46:18.586581 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.586584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.586727 (+ 143us) server_negotiation.cc:407] Connection header received
0805 22:46:18.586814 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.586814 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.586825 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.586896 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:18.587050 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.587522 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.588457 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.588955 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.589040 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.589049 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.589201 (+ 152us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.589248 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:46:18.589541 20637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f931be7dc6408cb7f880f0f3b7dc28" candidate_uuid: "781e922634c341e1a8a36384bd32f9e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c539dce7388644b297567440a89aa79a" is_pre_election: true
I20260805 22:46:18.589592 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.586469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:41631 (local address 127.23.197.66:34855)
0805 22:46:18.586647 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:18.586655 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:18.586741 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:18.586897 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:18.586898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:18.586900 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:18.586975 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.586976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.587608 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.587609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.588362 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:18.588364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.589039 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.589040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:18.589120 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.589124 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:18.589193 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 22:46:18.589538 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":117,"threads_started":1}
I20260805 22:46:18.589704 20637 raft_consensus.cc:2454] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 781e922634c341e1a8a36384bd32f9e6 in term 0.
I20260805 22:46:18.589919 20728 leader_election.cc:302] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 781e922634c341e1a8a36384bd32f9e6, c539dce7388644b297567440a89aa79a; no voters:
I20260805 22:46:18.589968 21029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f931be7dc6408cb7f880f0f3b7dc28" candidate_uuid: "781e922634c341e1a8a36384bd32f9e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8025628a656b4de09dab8037d49adb8a" is_pre_election: true
I20260805 22:46:18.590039 21135 raft_consensus.cc:2781] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:46:18.590096 21135 raft_consensus.cc:482] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:46:18.590121 21135 raft_consensus.cc:3037] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:18.590111 21029 raft_consensus.cc:2454] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 781e922634c341e1a8a36384bd32f9e6 in term 0.
I20260805 22:46:18.590921 21135 raft_consensus.cc:504] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } }
I20260805 22:46:18.591069 21135 leader_election.cc:288] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [CANDIDATE]: Term 1 election: Requested vote from peers c539dce7388644b297567440a89aa79a (127.23.197.65:43775), 8025628a656b4de09dab8037d49adb8a (127.23.197.67:41631)
I20260805 22:46:18.591218 21029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f931be7dc6408cb7f880f0f3b7dc28" candidate_uuid: "781e922634c341e1a8a36384bd32f9e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8025628a656b4de09dab8037d49adb8a"
I20260805 22:46:18.591298 21029 raft_consensus.cc:3037] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:18.591265 20637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f931be7dc6408cb7f880f0f3b7dc28" candidate_uuid: "781e922634c341e1a8a36384bd32f9e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c539dce7388644b297567440a89aa79a"
I20260805 22:46:18.591378 20637 raft_consensus.cc:3037] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:18.592032 21029 raft_consensus.cc:2454] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 781e922634c341e1a8a36384bd32f9e6 in term 1.
I20260805 22:46:18.592163 20637 raft_consensus.cc:2454] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 781e922634c341e1a8a36384bd32f9e6 in term 1.
I20260805 22:46:18.592288 20727 leader_election.cc:302] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 781e922634c341e1a8a36384bd32f9e6, 8025628a656b4de09dab8037d49adb8a; no voters:
I20260805 22:46:18.592432 21135 raft_consensus.cc:2781] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:18.592622 21135 raft_consensus.cc:686] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 1 LEADER]: Becoming Leader. State: Replica: 781e922634c341e1a8a36384bd32f9e6, State: Running, Role: LEADER
I20260805 22:46:18.592738 21135 consensus_queue.cc:237] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } }
I20260805 22:46:18.593750 20411 catalog_manager.cc:5495] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 781e922634c341e1a8a36384bd32f9e6 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "781e922634c341e1a8a36384bd32f9e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "781e922634c341e1a8a36384bd32f9e6" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 37005 } health_report { overall_health: HEALTHY } } }
W20260805 22:46:18.598129 20894 tablet.cc:2369] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:46:18.608325 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.606142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.23.197.66:37005)
0805 22:46:18.606227 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.606229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.606237 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:18.606273 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.606273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.606289 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.606353 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:18.606472 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.606925 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.607666 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.607867 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.607927 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.608061 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:18.608153 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:18.608220 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.608237 (+ 17us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.608255 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 22:46:18.610112 20637 raft_consensus.cc:1261] T 95f931be7dc6408cb7f880f0f3b7dc28 P c539dce7388644b297567440a89aa79a [term 1 FOLLOWER]: Refusing update from remote peer 781e922634c341e1a8a36384bd32f9e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:18.610114 21029 raft_consensus.cc:1261] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [term 1 FOLLOWER]: Refusing update from remote peer 781e922634c341e1a8a36384bd32f9e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:18.610524 21139 consensus_queue.cc:1035] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8025628a656b4de09dab8037d49adb8a" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:18.610662 21135 consensus_queue.cc:1035] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c539dce7388644b297567440a89aa79a" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:18.612986 21147 mvcc.cc:205] Tried to move back new op lower bound from 7315333032384323584 to 7315333032316825600. Current Snapshot: MvccSnapshot[applied={T|T < 7315333032384323584}]
I20260805 22:46:18.613517 21148 mvcc.cc:205] Tried to move back new op lower bound from 7315333032384323584 to 7315333032316825600. Current Snapshot: MvccSnapshot[applied={T|T < 7315333032384323584}]
I20260805 22:46:18.615523 20412 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:18.615629 20412 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:18.616432 20412 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f86bfe66a1b4ddfaea32c1c4d072f45: Sending DeleteTablet for 3 replicas of tablet 95f931be7dc6408cb7f880f0f3b7dc28
I20260805 22:46:18.616791 20604 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f931be7dc6408cb7f880f0f3b7dc28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:18 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:52694
I20260805 22:46:18.616856 20805 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f931be7dc6408cb7f880f0f3b7dc28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:18 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:59686
I20260805 22:46:18.617069 21167 tablet_replica.cc:330] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: stopping tablet replica
I20260805 22:46:18.617056 20997 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f931be7dc6408cb7f880f0f3b7dc28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:18 UTC) from {username='kudu', principal='kudu/127.23.197.126@KRBTEST.COM'} at 127.0.0.1:45466
I20260805 22:46:18.617173 21167 raft_consensus.cc:2229] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:46:18.617287 21167 raft_consensus.cc:2258] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:18.617306 21168 tablet_replica.cc:330] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: stopping tablet replica
I20260805 22:46:18.617452 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 20506
I20260805 22:46:18.617491 21168 raft_consensus.cc:2229] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:46:18.617702 21168 raft_consensus.cc:2258] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:18.620368 21168 ts_tablet_manager.cc:1905] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:18.622517 21168 ts_tablet_manager.cc:1918] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:18.622679 21168 log.cc:1196] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/95f931be7dc6408cb7f880f0f3b7dc28
I20260805 22:46:18.623054 21168 ts_tablet_manager.cc:1939] T 95f931be7dc6408cb7f880f0f3b7dc28 P 8025628a656b4de09dab8037d49adb8a: Deleting consensus metadata
I20260805 22:46:18.623590 20392 catalog_manager.cc:4841] TS 8025628a656b4de09dab8037d49adb8a (127.23.197.67:41631): tablet 95f931be7dc6408cb7f880f0f3b7dc28 (table test-table [id=605dd2aedf52416ca214fa18b388b5e0]) successfully deleted
I20260805 22:46:18.625434 21167 ts_tablet_manager.cc:1905] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:18.627558 21167 ts_tablet_manager.cc:1918] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:18.627646 21167 log.cc:1196] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/95f931be7dc6408cb7f880f0f3b7dc28
I20260805 22:46:18.628028 21167 ts_tablet_manager.cc:1939] T 95f931be7dc6408cb7f880f0f3b7dc28 P 781e922634c341e1a8a36384bd32f9e6: Deleting consensus metadata
I20260805 22:46:18.628553 20393 catalog_manager.cc:4841] TS 781e922634c341e1a8a36384bd32f9e6 (127.23.197.66:37005): tablet 95f931be7dc6408cb7f880f0f3b7dc28 (table test-table [id=605dd2aedf52416ca214fa18b388b5e0]) successfully deleted
W20260805 22:46:18.635439 20393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:43775 (error 108)
I20260805 22:46:18.636034 21116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.635886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:43775 (local address 127.0.0.1:52702)
0805 22:46:18.635953 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:18.635996 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:43775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260805 22:46:18.636114 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 20705
W20260805 22:46:18.641052 20393 catalog_manager.cc:4584] TS c539dce7388644b297567440a89aa79a (127.23.197.65:43775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95f931be7dc6408cb7f880f0f3b7dc28: Network error: Client connection negotiation failed: client connection to 127.23.197.65:43775: connect: Connection refused (error 111)
I20260805 22:46:18.644716 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 20903
I20260805 22:46:18.669489 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 20364
2026-08-05T22:46:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2199 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-05T22:46:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:18Z Disabled control of system clock
I20260805 22:46:18.704869 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36791
--webserver_interface=127.23.197.126
--webserver_port=0
--builtin_ntp_servers=127.23.197.84:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:36791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 22:46:18.790004 21179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:18.790230 21179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:18.790251 21179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:18.791366 21179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:18.791401 21179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:18.791414 21179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:18.791426 21179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:18.791438 21179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:18.792755 21179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:36791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:18.792982 21179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:18.793228 21179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:18.798457 21185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.798465 21184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.798465 21187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:18.798805 21179 server_base.cc:1034] running on GCE node
I20260805 22:46:18.799089 21179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:18.799369 21179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:18.800611 21179 hybrid_clock.cc:648] HybridClock initialized: now 1785969978800595 us; error 35 us; skew 500 ppm
I20260805 22:46:18.802134 21179 webserver.cc:460] Webserver started at http://127.23.197.126:43197/ using document root <none> and password file <none>
I20260805 22:46:18.802444 21179 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:18.802523 21179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:18.802688 21179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:18.803802 21179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "1601c1303837f6af35c4c6df68031d43"
server_key_iv: "d97399e34ce08cb4cdf198cf5ce16dca"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.804196 21179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "1601c1303837f6af35c4c6df68031d43"
server_key_iv: "d97399e34ce08cb4cdf198cf5ce16dca"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.805769 21179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:18.806682 21197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.806898 21179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:18.807017 21179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "1601c1303837f6af35c4c6df68031d43"
server_key_iv: "d97399e34ce08cb4cdf198cf5ce16dca"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.807106 21179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:18.828337 21179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:18.828855 21179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:18.829037 21179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:18.833680 21179 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:36791
I20260805 22:46:18.833757 21278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:36791 every 8 connection(s)
I20260805 22:46:18.834128 21179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:18.835263 21279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:18.837942 21279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8: Bootstrap starting.
I20260805 22:46:18.838649 21279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:18.839005 21279 log.cc:824] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:18.839896 21279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8: No bootstrap required, opened a new log
I20260805 22:46:18.840241 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 21179
I20260805 22:46:18.840365 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:18.840574 24341 external_mini_cluster.cc:1428] Setting key 3c2beb1a121ddc851feeecf542293769
I20260805 22:46:18.842550 21279 raft_consensus.cc:348] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36791 } }
I20260805 22:46:18.842760 21279 raft_consensus.cc:374] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:18.842814 21279 raft_consensus.cc:729] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d289b33b9cf44cf58ec2c31cc30ea7a8, State: Initialized, Role: FOLLOWER
I20260805 22:46:18.842962 21279 consensus_queue.cc:260] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36791 } }
I20260805 22:46:18.843073 21279 raft_consensus.cc:388] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:18.843123 21279 raft_consensus.cc:482] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:18.843173 21279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:18.843804 21279 raft_consensus.cc:504] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36791 } }
I20260805 22:46:18.843935 21279 leader_election.cc:302] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d289b33b9cf44cf58ec2c31cc30ea7a8; no voters:
I20260805 22:46:18.844157 21279 leader_election.cc:288] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:18.844197 21284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:18.844338 21284 raft_consensus.cc:686] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [term 1 LEADER]: Becoming Leader. State: Replica: d289b33b9cf44cf58ec2c31cc30ea7a8, State: Running, Role: LEADER
I20260805 22:46:18.844424 21284 consensus_queue.cc:237] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36791 } }
I20260805 22:46:18.844471 21279 sys_catalog.cc:564] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:18.844733 21284 sys_catalog.cc:455] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d289b33b9cf44cf58ec2c31cc30ea7a8. Latest consensus state: current_term: 1 leader_uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36791 } } }
I20260805 22:46:18.844746 21288 sys_catalog.cc:455] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d289b33b9cf44cf58ec2c31cc30ea7a8" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36791 } } }
I20260805 22:46:18.844902 21288 sys_catalog.cc:458] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:18.844897 21284 sys_catalog.cc:458] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:18.845216 21296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:18.845744 21296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:18.847838 21296 catalog_manager.cc:1347] Generated new cluster ID: d45bf97930a24a7196328333ec8068c6
I20260805 22:46:18.847898 21296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:18.848186 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:18.841258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40168 (local address 127.23.197.126:36791)
0805 22:46:18.841506 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:18.841514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:18.841543 (+ 29us) server_negotiation.cc:407] Connection header received
0805 22:46:18.842053 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:18.842062 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:18.842126 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:18.842257 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:18.842525 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.844373 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.845680 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:18.846323 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:18.846479 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:18.847425 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:18.847429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:18.847429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:18.847641 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:18.847684 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:18.847700 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:18.847773 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:18.847884 (+ 111us) server_negotiation.cc:299] Negotiation successful
0805 22:46:18.847966 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260805 22:46:18.876729 21296 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:18.877419 21296 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:18.883091 21296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8: Generated new TSK 0
I20260805 22:46:18.883324 21296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:18.888687 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.65:0
--local_ip_for_outbound_sockets=127.23.197.65
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36791
--builtin_ntp_servers=127.23.197.84:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 22:46:18.974566 21318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:18.974781 21318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:18.974802 21318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:18.975980 21318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:18.976024 21318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:18.976050 21318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.65
I20260805 22:46:18.977491 21318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.197.65
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.65
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:18.977769 21318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:18.978111 21318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:18.983536 21332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.983520 21329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:18.983538 21328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:18.983829 21318 server_base.cc:1034] running on GCE node
I20260805 22:46:18.984052 21318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:18.984301 21318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:18.985472 21318 hybrid_clock.cc:648] HybridClock initialized: now 1785969978985444 us; error 41 us; skew 500 ppm
I20260805 22:46:18.987116 21318 webserver.cc:460] Webserver started at http://127.23.197.65:46713/ using document root <none> and password file <none>
I20260805 22:46:18.987447 21318 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:18.987531 21318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:18.987710 21318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:18.988974 21318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/data/instance:
uuid: "07229c1b5b1748a0a5d0d83f24634619"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "a085e5ed116c8584e814e5c72f533865"
server_key_iv: "0decefd119defb5830e99a3555852afe"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.989367 21318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance:
uuid: "07229c1b5b1748a0a5d0d83f24634619"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "a085e5ed116c8584e814e5c72f533865"
server_key_iv: "0decefd119defb5830e99a3555852afe"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.990869 21318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:46:18.991767 21346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:18.992009 21318 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:18.992127 21318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/wal
uuid: "07229c1b5b1748a0a5d0d83f24634619"
format_stamp: "Formatted at 2026-08-05 22:46:18 on dist-test-slave-g170"
server_key: "a085e5ed116c8584e814e5c72f533865"
server_key_iv: "0decefd119defb5830e99a3555852afe"
server_key_version: "encryptionkey@0"
I20260805 22:46:18.992219 21318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:19.006471 21318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:19.006851 21318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:19.007023 21318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:19.007294 21318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:19.007670 21318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:19.007730 21318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:19.007777 21318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:19.007838 21318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:19.014848 21318 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.65:33957
I20260805 22:46:19.014917 21510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.65:33957 every 8 connection(s)
I20260805 22:46:19.015316 21318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/data/info.pb
I20260805 22:46:19.021102 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.016248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.65:57787 (local address 127.23.197.126:36791)
0805 22:46:19.016325 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.016327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.016590 (+ 263us) server_negotiation.cc:407] Connection header received
0805 22:46:19.017210 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.017212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.017225 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.017280 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:19.018007 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.018555 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.019575 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.019810 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.019892 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.020807 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:19.020809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:19.020809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:19.020815 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:19.020830 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:19.020835 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:19.020885 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.020981 (+ 96us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.021017 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 22:46:19.021376 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.016229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36791 (local address 127.23.197.65:57787)
0805 22:46:19.016457 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:19.016494 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:19.017097 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:19.017373 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:19.017377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:19.017385 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:19.017911 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.017915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.018660 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.018662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.019462 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.019464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.019903 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.019904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.020043 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.020628 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:46:19.020628 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:19.020659 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:46:19.020683 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:19.020885 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:19.020891 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:19.020895 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:19.020985 (+ 90us) client_negotiation.cc:240] Negotiation successful
0805 22:46:19.021095 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260805 22:46:19.022101 21511 heartbeater.cc:347] Connected to a master server at 127.23.197.126:36791
I20260805 22:46:19.022255 21511 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:19.022488 21511 heartbeater.cc:511] Master 127.23.197.126:36791 requested a full tablet report, sending...
I20260805 22:46:19.023074 21224 ts_manager.cc:194] Registered new tserver with Master: 07229c1b5b1748a0a5d0d83f24634619 (127.23.197.65:33957)
I20260805 22:46:19.023722 21224 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.65:57787
I20260805 22:46:19.025394 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 21318
I20260805 22:46:19.025566 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-0/wal/instance
I20260805 22:46:19.025856 24341 external_mini_cluster.cc:1428] Setting key 8aafcfc73b46afaec23ecfed0579124f
I20260805 22:46:19.027007 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.66:0
--local_ip_for_outbound_sockets=127.23.197.66
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36791
--builtin_ntp_servers=127.23.197.84:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 22:46:19.110320 21514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:19.110530 21514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:19.110553 21514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:19.111757 21514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:19.111801 21514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:19.111827 21514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.66
I20260805 22:46:19.113276 21514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.197.66
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.66
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:19.113528 21514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:19.113842 21514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:19.118930 21521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:19.118937 21524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:19.118965 21514 server_base.cc:1034] running on GCE node
W20260805 22:46:19.118961 21522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:19.119414 21514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:19.119688 21514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:19.120859 21514 hybrid_clock.cc:648] HybridClock initialized: now 1785969979120836 us; error 37 us; skew 500 ppm
I20260805 22:46:19.122146 21514 webserver.cc:460] Webserver started at http://127.23.197.66:34147/ using document root <none> and password file <none>
I20260805 22:46:19.122432 21514 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:19.122511 21514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:19.122673 21514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:19.123890 21514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/data/instance:
uuid: "18b3e07db5b3416d94624b65e0025706"
format_stamp: "Formatted at 2026-08-05 22:46:19 on dist-test-slave-g170"
server_key: "3d62f051d08e2fd6ea0aa6699a82b38a"
server_key_iv: "def64bc22aef5161ae851f3bf9718d77"
server_key_version: "encryptionkey@0"
I20260805 22:46:19.124301 21514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance:
uuid: "18b3e07db5b3416d94624b65e0025706"
format_stamp: "Formatted at 2026-08-05 22:46:19 on dist-test-slave-g170"
server_key: "3d62f051d08e2fd6ea0aa6699a82b38a"
server_key_iv: "def64bc22aef5161ae851f3bf9718d77"
server_key_version: "encryptionkey@0"
I20260805 22:46:19.125875 21514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:19.126724 21530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:19.126955 21514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:19.127076 21514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/wal
uuid: "18b3e07db5b3416d94624b65e0025706"
format_stamp: "Formatted at 2026-08-05 22:46:19 on dist-test-slave-g170"
server_key: "3d62f051d08e2fd6ea0aa6699a82b38a"
server_key_iv: "def64bc22aef5161ae851f3bf9718d77"
server_key_version: "encryptionkey@0"
I20260805 22:46:19.127158 21514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:19.156713 21514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:19.157102 21514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:19.157277 21514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:19.157554 21514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:19.157924 21514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:19.157991 21514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:19.158041 21514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:19.158067 21514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:19.163995 21514 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.66:43871
I20260805 22:46:19.164045 21699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.66:43871 every 8 connection(s)
I20260805 22:46:19.164407 21514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/data/info.pb
I20260805 22:46:19.170039 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.165272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:53359 (local address 127.23.197.126:36791)
0805 22:46:19.165363 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.165365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.165698 (+ 333us) server_negotiation.cc:407] Connection header received
0805 22:46:19.166212 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.166214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.166226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.166301 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:19.167081 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.167598 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.168673 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.168882 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.168936 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.169807 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:19.169808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:19.169808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:19.169813 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:19.169824 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:19.169828 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:19.169868 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.169948 (+ 80us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.169975 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 22:46:19.170303 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.165242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36791 (local address 127.23.197.66:53359)
0805 22:46:19.165500 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:19.165546 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:19.166082 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:19.166414 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:19.166418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:19.166427 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:19.166964 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.166969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.167736 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.167737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.168587 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.168590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.168967 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.168968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.169098 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.169672 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:46:19.169672 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:19.169703 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:46:19.169728 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:19.169878 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:19.169881 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:19.169883 (+ 2us) client_negotiation.cc:769] Sending connection context
0805 22:46:19.169939 (+ 56us) client_negotiation.cc:240] Negotiation successful
0805 22:46:19.170024 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":148,"threads_started":1}
I20260805 22:46:19.170974 21701 heartbeater.cc:347] Connected to a master server at 127.23.197.126:36791
I20260805 22:46:19.171072 21701 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:19.171281 21701 heartbeater.cc:511] Master 127.23.197.126:36791 requested a full tablet report, sending...
I20260805 22:46:19.171801 21224 ts_manager.cc:194] Registered new tserver with Master: 18b3e07db5b3416d94624b65e0025706 (127.23.197.66:43871)
I20260805 22:46:19.172200 21224 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.66:53359
I20260805 22:46:19.173243 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 21514
I20260805 22:46:19.173345 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/wal/instance
I20260805 22:46:19.173509 24341 external_mini_cluster.cc:1428] Setting key 1748da7bfaa405fcc0208c43b0a899a0
I20260805 22:46:19.174489 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.197.67:0
--local_ip_for_outbound_sockets=127.23.197.67
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36791
--builtin_ntp_servers=127.23.197.84:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 22:46:19.260360 21705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:19.260564 21705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:19.260586 21705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:19.261811 21705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:19.261854 21705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:19.261880 21705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.197.67
I20260805 22:46:19.263231 21705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.197.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.197.67
--webserver_port=0
--tserver_master_addrs=127.23.197.126:36791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.197.67
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:19.263489 21705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:19.263809 21705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:19.269147 21711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:19.269160 21712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:19.269181 21714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:19.269840 21705 server_base.cc:1034] running on GCE node
I20260805 22:46:19.270061 21705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:19.270334 21705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:19.271492 21705 hybrid_clock.cc:648] HybridClock initialized: now 1785969979271477 us; error 32 us; skew 500 ppm
I20260805 22:46:19.272953 21705 webserver.cc:460] Webserver started at http://127.23.197.67:46329/ using document root <none> and password file <none>
I20260805 22:46:19.273247 21705 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:19.273324 21705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:19.273479 21705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:19.274549 21705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/data/instance:
uuid: "33e35d4099a04a0abc5677f18ccb37c5"
format_stamp: "Formatted at 2026-08-05 22:46:19 on dist-test-slave-g170"
server_key: "ebdb86445519296e84a8d94b56f648f6"
server_key_iv: "06bbc0376f0d8cf13ba1f3a15b4d2cec"
server_key_version: "encryptionkey@0"
I20260805 22:46:19.274922 21705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance:
uuid: "33e35d4099a04a0abc5677f18ccb37c5"
format_stamp: "Formatted at 2026-08-05 22:46:19 on dist-test-slave-g170"
server_key: "ebdb86445519296e84a8d94b56f648f6"
server_key_iv: "06bbc0376f0d8cf13ba1f3a15b4d2cec"
server_key_version: "encryptionkey@0"
I20260805 22:46:19.276288 21705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:19.277088 21723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:19.277275 21705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:19.277381 21705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/wal
uuid: "33e35d4099a04a0abc5677f18ccb37c5"
format_stamp: "Formatted at 2026-08-05 22:46:19 on dist-test-slave-g170"
server_key: "ebdb86445519296e84a8d94b56f648f6"
server_key_iv: "06bbc0376f0d8cf13ba1f3a15b4d2cec"
server_key_version: "encryptionkey@0"
I20260805 22:46:19.277456 21705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:19.287516 21705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:19.287851 21705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:19.288015 21705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:19.288264 21705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 22:46:19.288614 21705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 22:46:19.288671 21705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:19.288713 21705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 22:46:19.288741 21705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:19.295790 21705 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.67:41823
I20260805 22:46:19.295847 21884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.67:41823 every 8 connection(s)
I20260805 22:46:19.296257 21705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/data/info.pb
I20260805 22:46:19.300887 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 21705
I20260805 22:46:19.301968 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.297068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.67:41009 (local address 127.23.197.126:36791)
0805 22:46:19.297156 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.297159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.297381 (+ 222us) server_negotiation.cc:407] Connection header received
0805 22:46:19.297880 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.297881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.297892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.297950 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:19.298629 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.299322 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.300317 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.300507 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.300578 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.301573 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:19.301574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:19.301575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:19.301581 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:19.301602 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:19.301607 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:19.301680 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.301776 (+ 96us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.301809 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 22:46:19.302124 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.297035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:36791 (local address 127.23.197.67:41009)
0805 22:46:19.297236 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:19.297273 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:19.297787 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:19.298005 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:19.298008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:19.298017 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:19.298549 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.298554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.299424 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.299425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.300218 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.300220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.300595 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.300596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.300706 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.301395 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 22:46:19.301395 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:19.301434 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 22:46:19.301467 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:19.301681 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:19.301687 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:19.301690 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:19.301766 (+ 76us) client_negotiation.cc:240] Negotiation successful
0805 22:46:19.301883 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":110,"threads_started":1}
I20260805 22:46:19.302205 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/wal/instance
I20260805 22:46:19.302474 24341 external_mini_cluster.cc:1428] Setting key c1f1ac6e7f330344ae82f3617cdc62dc
I20260805 22:46:19.302616 21885 heartbeater.cc:347] Connected to a master server at 127.23.197.126:36791
I20260805 22:46:19.302716 21885 heartbeater.cc:464] Registering TS with master...
I20260805 22:46:19.302946 21885 heartbeater.cc:511] Master 127.23.197.126:36791 requested a full tablet report, sending...
I20260805 22:46:19.303357 21219 ts_manager.cc:194] Registered new tserver with Master: 33e35d4099a04a0abc5677f18ccb37c5 (127.23.197.67:41823)
I20260805 22:46:19.303750 21219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.197.67:41009
I20260805 22:46:19.314489 24341 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 22:46:19.318909 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.316082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.23.197.126:36791)
0805 22:46:19.316158 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.316160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.316223 (+ 63us) server_negotiation.cc:407] Connection header received
0805 22:46:19.316280 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.316280 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.316289 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.316335 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:19.316510 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.317077 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.317826 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.318007 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.318061 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.318716 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:19.318717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:19.318717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:19.318721 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:19.318733 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:19.318737 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:19.318780 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.318829 (+ 49us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.318846 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 22:46:19.320556 21219 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 22:46:19.322182 21219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 22:46:19.328991 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.324651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.23.197.65:33957)
0805 22:46:19.324920 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.324926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.325330 (+ 404us) server_negotiation.cc:407] Connection header received
0805 22:46:19.325385 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.325387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.325405 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.325502 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:19.325700 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.326499 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.327784 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.328542 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.328613 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.328701 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.328870 (+ 169us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.328908 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":139,"threads_started":1}
I20260805 22:46:19.328990 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.324515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33957 (local address 127.0.0.1:55776)
0805 22:46:19.325164 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:19.325174 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:19.325264 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:19.325516 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:19.325519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:19.325521 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:19.325584 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.325586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.326612 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.326613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.327675 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.327678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.328628 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.328629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.328710 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.328725 (+ 15us) client_negotiation.cc:769] Sending connection context
0805 22:46:19.328855 (+ 130us) client_negotiation.cc:240] Negotiation successful
0805 22:46:19.328910 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260805 22:46:19.329528 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.324717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.23.197.66:43871)
0805 22:46:19.325089 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.325092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.325102 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:46:19.325409 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.325412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.325429 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.325509 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:19.325838 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.327074 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.328393 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.329151 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.329239 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.329325 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.329387 (+ 62us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.329423 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":120,"threads_started":1}
I20260805 22:46:19.329600 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.324733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:41823 (local address 127.0.0.1:41708)
0805 22:46:19.325218 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:19.325228 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:19.325276 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:19.325632 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:19.325633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:19.325635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:19.325685 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.325687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.327209 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.327210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.328344 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.328347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.329411 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.329411 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.329461 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.329464 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:19.329510 (+ 46us) client_negotiation.cc:240] Negotiation successful
0805 22:46:19.329546 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":104,"threads_started":1}
I20260805 22:46:19.329497 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.324515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.66:43871 (local address 127.0.0.1:33522)
0805 22:46:19.324928 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:19.324958 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:19.325036 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:19.325623 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:19.325624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:19.325626 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:19.325697 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.325699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.327166 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.327168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.328298 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.328301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.329251 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.329252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.329320 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.329323 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:19.329376 (+ 53us) client_negotiation.cc:240] Negotiation successful
0805 22:46:19.329418 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":76,"thread_start_us":77,"threads_started":1}
I20260805 22:46:19.329874 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.324894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.23.197.67:41823)
0805 22:46:19.325098 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.325103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.325271 (+ 168us) server_negotiation.cc:407] Connection header received
0805 22:46:19.325447 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.325449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.325482 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.325557 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:19.325742 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.326941 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.328447 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.329219 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.329288 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.329378 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.329758 (+ 380us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.329814 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260805 22:46:19.330605 21794 tablet_service.cc:1467] Processing CreateTablet for tablet a32a92f3ab01406f982cc00d2ec32ada (DEFAULT_TABLE table=test-table [id=1fa64be9abd14372bc242681c579a87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:19.330607 21416 tablet_service.cc:1467] Processing CreateTablet for tablet a32a92f3ab01406f982cc00d2ec32ada (DEFAULT_TABLE table=test-table [id=1fa64be9abd14372bc242681c579a87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:19.330713 21600 tablet_service.cc:1467] Processing CreateTablet for tablet a32a92f3ab01406f982cc00d2ec32ada (DEFAULT_TABLE table=test-table [id=1fa64be9abd14372bc242681c579a87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 22:46:19.331008 21794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a32a92f3ab01406f982cc00d2ec32ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:19.331008 21416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a32a92f3ab01406f982cc00d2ec32ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:19.331099 21600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a32a92f3ab01406f982cc00d2ec32ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:19.334218 21905 tablet_bootstrap.cc:492] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: Bootstrap starting.
I20260805 22:46:19.334605 21908 tablet_bootstrap.cc:492] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: Bootstrap starting.
I20260805 22:46:19.335024 21905 tablet_bootstrap.cc:654] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:19.335179 21908 tablet_bootstrap.cc:654] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:19.335400 21905 log.cc:824] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:19.335516 21908 log.cc:824] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:19.335891 21907 tablet_bootstrap.cc:492] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619: Bootstrap starting.
I20260805 22:46:19.336249 21905 tablet_bootstrap.cc:492] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: No bootstrap required, opened a new log
I20260805 22:46:19.336338 21905 ts_tablet_manager.cc:1397] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 22:46:19.336541 21907 tablet_bootstrap.cc:654] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:19.336964 21907 log.cc:824] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:19.337174 21908 tablet_bootstrap.cc:492] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: No bootstrap required, opened a new log
I20260805 22:46:19.337275 21908 ts_tablet_manager.cc:1397] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260805 22:46:19.337915 21907 tablet_bootstrap.cc:492] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619: No bootstrap required, opened a new log
I20260805 22:46:19.337984 21907 ts_tablet_manager.cc:1397] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260805 22:46:19.339046 21905 raft_consensus.cc:348] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } }
I20260805 22:46:19.339218 21905 raft_consensus.cc:374] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:19.339257 21905 raft_consensus.cc:729] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b3e07db5b3416d94624b65e0025706, State: Initialized, Role: FOLLOWER
I20260805 22:46:19.339411 21905 consensus_queue.cc:260] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } }
I20260805 22:46:19.339454 21908 raft_consensus.cc:348] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } }
I20260805 22:46:19.339632 21908 raft_consensus.cc:374] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:19.339674 21905 ts_tablet_manager.cc:1428] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260805 22:46:19.339674 21908 raft_consensus.cc:729] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e35d4099a04a0abc5677f18ccb37c5, State: Initialized, Role: FOLLOWER
I20260805 22:46:19.339789 21701 heartbeater.cc:503] Master 127.23.197.126:36791 was elected leader, sending a full tablet report...
I20260805 22:46:19.339788 21908 consensus_queue.cc:260] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } }
I20260805 22:46:19.340101 21908 ts_tablet_manager.cc:1428] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:19.340229 21885 heartbeater.cc:503] Master 127.23.197.126:36791 was elected leader, sending a full tablet report...
I20260805 22:46:19.340821 21907 raft_consensus.cc:348] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } }
I20260805 22:46:19.340979 21907 raft_consensus.cc:374] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:19.341027 21907 raft_consensus.cc:729] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07229c1b5b1748a0a5d0d83f24634619, State: Initialized, Role: FOLLOWER
I20260805 22:46:19.341122 21907 consensus_queue.cc:260] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } }
I20260805 22:46:19.341367 21907 ts_tablet_manager.cc:1428] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:19.341468 21511 heartbeater.cc:503] Master 127.23.197.126:36791 was elected leader, sending a full tablet report...
W20260805 22:46:19.415153 21702 tablet.cc:2369] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 22:46:19.430984 21913 raft_consensus.cc:482] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 22:46:19.431152 21913 raft_consensus.cc:504] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } }
I20260805 22:46:19.431493 21913 leader_election.cc:288] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07229c1b5b1748a0a5d0d83f24634619 (127.23.197.65:33957), 33e35d4099a04a0abc5677f18ccb37c5 (127.23.197.67:41823)
I20260805 22:46:19.434844 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.431783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:50415 (local address 127.23.197.67:41823)
0805 22:46:19.431854 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.431857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.431903 (+ 46us) server_negotiation.cc:407] Connection header received
0805 22:46:19.431939 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.431940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.431951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.431998 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:19.432205 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.432850 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.433912 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.434500 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.434564 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.434572 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.434748 (+ 176us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.434785 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 22:46:19.434849 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.431682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.67:41823 (local address 127.23.197.66:50415)
0805 22:46:19.431732 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:19.431745 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:19.431830 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:19.432038 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:19.432040 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:19.432042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:19.432121 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.432123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.432928 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.432930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.433798 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.433801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.434587 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.434588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.434661 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.434665 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 22:46:19.434725 (+ 60us) client_negotiation.cc:240] Negotiation successful
0805 22:46:19.434768 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260805 22:46:19.435267 21916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.431682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33957 (local address 127.23.197.66:50515)
0805 22:46:19.432104 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:19.432119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:19.432194 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:19.432344 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:19.432345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:19.432347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 22:46:19.432416 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.432417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.433243 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.433245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.434269 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:19.434272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.434998 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.434999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:19.435071 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.435074 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 22:46:19.435151 (+ 77us) client_negotiation.cc:240] Negotiation successful
0805 22:46:19.435188 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":202,"thread_start_us":127,"threads_started":1}
I20260805 22:46:19.435256 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32a92f3ab01406f982cc00d2ec32ada" candidate_uuid: "18b3e07db5b3416d94624b65e0025706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33e35d4099a04a0abc5677f18ccb37c5" is_pre_election: true
I20260805 22:46:19.435319 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.431817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.197.66:50515 (local address 127.23.197.65:33957)
0805 22:46:19.431871 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.431873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.432189 (+ 316us) server_negotiation.cc:407] Connection header received
0805 22:46:19.432252 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.432253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.432268 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.432323 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 22:46:19.432492 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.433124 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.434370 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.434875 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.434944 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.434954 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.435198 (+ 244us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.435252 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260805 22:46:19.435431 21820 raft_consensus.cc:2454] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18b3e07db5b3416d94624b65e0025706 in term 0.
I20260805 22:46:19.435667 21441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32a92f3ab01406f982cc00d2ec32ada" candidate_uuid: "18b3e07db5b3416d94624b65e0025706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07229c1b5b1748a0a5d0d83f24634619" is_pre_election: true
I20260805 22:46:19.435779 21536 leader_election.cc:302] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b3e07db5b3416d94624b65e0025706, 33e35d4099a04a0abc5677f18ccb37c5; no voters:
I20260805 22:46:19.435798 21441 raft_consensus.cc:2454] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18b3e07db5b3416d94624b65e0025706 in term 0.
I20260805 22:46:19.436018 21913 raft_consensus.cc:2781] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 22:46:19.436075 21913 raft_consensus.cc:482] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 22:46:19.436093 21913 raft_consensus.cc:3037] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:19.436942 21913 raft_consensus.cc:504] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } }
I20260805 22:46:19.437101 21913 leader_election.cc:288] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [CANDIDATE]: Term 1 election: Requested vote from peers 07229c1b5b1748a0a5d0d83f24634619 (127.23.197.65:33957), 33e35d4099a04a0abc5677f18ccb37c5 (127.23.197.67:41823)
I20260805 22:46:19.437276 21441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32a92f3ab01406f982cc00d2ec32ada" candidate_uuid: "18b3e07db5b3416d94624b65e0025706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07229c1b5b1748a0a5d0d83f24634619"
I20260805 22:46:19.437343 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32a92f3ab01406f982cc00d2ec32ada" candidate_uuid: "18b3e07db5b3416d94624b65e0025706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33e35d4099a04a0abc5677f18ccb37c5"
I20260805 22:46:19.437433 21820 raft_consensus.cc:3037] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:19.437398 21441 raft_consensus.cc:3037] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:19.438217 21441 raft_consensus.cc:2454] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18b3e07db5b3416d94624b65e0025706 in term 1.
I20260805 22:46:19.438230 21820 raft_consensus.cc:2454] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18b3e07db5b3416d94624b65e0025706 in term 1.
I20260805 22:46:19.438450 21531 leader_election.cc:302] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07229c1b5b1748a0a5d0d83f24634619, 18b3e07db5b3416d94624b65e0025706; no voters:
I20260805 22:46:19.438616 21913 raft_consensus.cc:2781] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:19.438777 21913 raft_consensus.cc:686] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 1 LEADER]: Becoming Leader. State: Replica: 18b3e07db5b3416d94624b65e0025706, State: Running, Role: LEADER
I20260805 22:46:19.438894 21913 consensus_queue.cc:237] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } }
I20260805 22:46:19.439896 21224 catalog_manager.cc:5495] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18b3e07db5b3416d94624b65e0025706 (127.23.197.66). New cstate: current_term: 1 leader_uuid: "18b3e07db5b3416d94624b65e0025706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18b3e07db5b3416d94624b65e0025706" member_type: VOTER last_known_addr { host: "127.23.197.66" port: 43871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 } health_report { overall_health: UNKNOWN } } }
I20260805 22:46:19.473841 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.471123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33528 (local address 127.23.197.66:43871)
0805 22:46:19.471213 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:19.471215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:19.471224 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:19.471263 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:19.471264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:19.471294 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:19.471363 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:19.471476 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.471989 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.472997 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:19.473216 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:19.473278 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:19.473476 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:19.473582 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:19.473651 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:19.473731 (+ 80us) server_negotiation.cc:299] Negotiation successful
0805 22:46:19.473753 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 22:46:19.475970 21441 raft_consensus.cc:1261] T a32a92f3ab01406f982cc00d2ec32ada P 07229c1b5b1748a0a5d0d83f24634619 [term 1 FOLLOWER]: Refusing update from remote peer 18b3e07db5b3416d94624b65e0025706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:19.475997 21820 raft_consensus.cc:1261] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [term 1 FOLLOWER]: Refusing update from remote peer 18b3e07db5b3416d94624b65e0025706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:19.476425 21913 consensus_queue.cc:1035] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33e35d4099a04a0abc5677f18ccb37c5" member_type: VOTER last_known_addr { host: "127.23.197.67" port: 41823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:19.476600 21917 consensus_queue.cc:1035] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07229c1b5b1748a0a5d0d83f24634619" member_type: VOTER last_known_addr { host: "127.23.197.65" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:19.478875 21924 mvcc.cc:205] Tried to move back new op lower bound from 7315333035930537984 to 7315333035782373376. Current Snapshot: MvccSnapshot[applied={T|T < 7315333035930537984}]
I20260805 22:46:19.479041 21925 mvcc.cc:205] Tried to move back new op lower bound from 7315333035930537984 to 7315333035782373376. Current Snapshot: MvccSnapshot[applied={T|T < 7315333035930537984}]
I20260805 22:46:19.480111 21926 mvcc.cc:205] Tried to move back new op lower bound from 7315333035930537984 to 7315333035782373376. Current Snapshot: MvccSnapshot[applied={T|T < 7315333035930537984}]
I20260805 22:46:19.481804 21219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:19.481911 21219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 22:46:19.482769 21219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d289b33b9cf44cf58ec2c31cc30ea7a8: Sending DeleteTablet for 3 replicas of tablet a32a92f3ab01406f982cc00d2ec32ada
I20260805 22:46:19.483106 21416 tablet_service.cc:1514] Processing DeleteTablet for tablet a32a92f3ab01406f982cc00d2ec32ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:19 UTC) from {username='slave'} at 127.0.0.1:55776
I20260805 22:46:19.483249 21794 tablet_service.cc:1514] Processing DeleteTablet for tablet a32a92f3ab01406f982cc00d2ec32ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:19 UTC) from {username='slave'} at 127.0.0.1:41708
I20260805 22:46:19.483547 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 21318
I20260805 22:46:19.493240 21600 tablet_service.cc:1514] Processing DeleteTablet for tablet a32a92f3ab01406f982cc00d2ec32ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 22:46:19 UTC) from {username='slave'} at 127.0.0.1:33522
I20260805 22:46:19.493988 21938 tablet_replica.cc:330] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: stopping tablet replica
I20260805 22:46:19.494151 21938 raft_consensus.cc:2229] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 22:46:19.494297 21938 raft_consensus.cc:2258] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:19.494808 21939 tablet_replica.cc:330] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: stopping tablet replica
I20260805 22:46:19.494939 21939 raft_consensus.cc:2229] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 1 LEADER]: Raft consensus shutting down.
I20260805 22:46:19.495087 21939 raft_consensus.cc:2258] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 22:46:19.495155 21938 ts_tablet_manager.cc:1905] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:19.495580 21939 ts_tablet_manager.cc:1905] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 22:46:19.497138 21938 ts_tablet_manager.cc:1918] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:19.497236 21938 log.cc:1196] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-2/wal/wals/a32a92f3ab01406f982cc00d2ec32ada
I20260805 22:46:19.497535 21938 ts_tablet_manager.cc:1939] T a32a92f3ab01406f982cc00d2ec32ada P 33e35d4099a04a0abc5677f18ccb37c5: Deleting consensus metadata
I20260805 22:46:19.497731 21939 ts_tablet_manager.cc:1918] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 22:46:19.497813 21939 log.cc:1196] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: Deleting WAL directory at /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785969725087118-24341-0/minicluster-data/ts-1/wal/wals/a32a92f3ab01406f982cc00d2ec32ada
I20260805 22:46:19.498108 21939 ts_tablet_manager.cc:1939] T a32a92f3ab01406f982cc00d2ec32ada P 18b3e07db5b3416d94624b65e0025706: Deleting consensus metadata
I20260805 22:46:19.498637 21203 catalog_manager.cc:4841] TS 18b3e07db5b3416d94624b65e0025706 (127.23.197.66:43871): tablet a32a92f3ab01406f982cc00d2ec32ada (table test-table [id=1fa64be9abd14372bc242681c579a87e]) successfully deleted
I20260805 22:46:19.498780 21203 catalog_manager.cc:4841] TS 33e35d4099a04a0abc5677f18ccb37c5 (127.23.197.67:41823): tablet a32a92f3ab01406f982cc00d2ec32ada (table test-table [id=1fa64be9abd14372bc242681c579a87e]) successfully deleted
W20260805 22:46:19.500380 21200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.197.65:33957 (error 108)
I20260805 22:46:19.500973 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:19.500833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.65:33957 (local address 127.0.0.1:55792)
0805 22:46:19.500883 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:19.500934 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260805 22:46:19.501058 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 21514
W20260805 22:46:19.501102 21200 catalog_manager.cc:4584] TS 07229c1b5b1748a0a5d0d83f24634619 (127.23.197.65:33957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a32a92f3ab01406f982cc00d2ec32ada: Network error: Client connection negotiation failed: client connection to 127.23.197.65:33957: connect: Connection refused (error 111)
I20260805 22:46:19.518147 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 21705
I20260805 22:46:19.535221 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 21179
2026-08-05T22:46:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (865 ms)
[----------] 6 tests from EncryptionPolicyTest (9363 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:19 dist-test-slave-g170 krb5kdc[21943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:19 dist-test-slave-g170 krb5kdc[21943](info): set up 2 sockets
Aug 05 22:46:19 dist-test-slave-g170 krb5kdc[21943](info): commencing operation
krb5kdc: starting...
W20260805 22:46:20.812117 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:20 dist-test-slave-g170 krb5kdc[21943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:20.943792 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:21.029335 21964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:21.029544 21964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:21.029568 21964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:21.030740 21964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:21.030787 21964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:21.030802 21964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:21.030814 21964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:21.030826 21964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:21.032347 21964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43637
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:21.032606 21964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:21.032907 21964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:21.037824 21972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:21.037824 21969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:21.037840 21970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:21.038156 21964 server_base.cc:1034] running on GCE node
I20260805 22:46:21.038393 21964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:21.038677 21964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:21.039872 21964 hybrid_clock.cc:648] HybridClock initialized: now 1785969981039859 us; error 35 us; skew 500 ppm
Aug 05 22:46:21 dist-test-slave-g170 krb5kdc[21943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969981, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:21.042848 21964 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:21.043306 21964 webserver.cc:460] Webserver started at http://127.0.0.1:39205/ using document root <none> and password file <none>
I20260805 22:46:21.043509 21964 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:21.043558 21964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:21.043668 21964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:21.044888 21964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "fbc833fc8dc74c8f82bc7318f882f738"
format_stamp: "Formatted at 2026-08-05 22:46:21 on dist-test-slave-g170"
server_key: "d7223d9d2e19437f22f9dbce8de2a472"
server_key_iv: "a5f4c14cd052a6929adc325b58060952"
server_key_version: "encryptionkey@0"
I20260805 22:46:21.045264 21964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "fbc833fc8dc74c8f82bc7318f882f738"
format_stamp: "Formatted at 2026-08-05 22:46:21 on dist-test-slave-g170"
server_key: "d7223d9d2e19437f22f9dbce8de2a472"
server_key_iv: "a5f4c14cd052a6929adc325b58060952"
server_key_version: "encryptionkey@0"
I20260805 22:46:21.046802 21964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:46:21.047631 21982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:21.047901 21964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:21.048051 21964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "fbc833fc8dc74c8f82bc7318f882f738"
format_stamp: "Formatted at 2026-08-05 22:46:21 on dist-test-slave-g170"
server_key: "d7223d9d2e19437f22f9dbce8de2a472"
server_key_iv: "a5f4c14cd052a6929adc325b58060952"
server_key_version: "encryptionkey@0"
I20260805 22:46:21.048154 21964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:21.063333 21964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:21.065881 21964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:21.066061 21964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:21.070449 21964 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43637
I20260805 22:46:21.070492 22056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43637 every 8 connection(s)
I20260805 22:46:21.070873 21964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:21.071820 22057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:21.074381 22057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738: Bootstrap starting.
I20260805 22:46:21.075066 22057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:21.075455 22057 log.cc:824] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:21.076406 22057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738: No bootstrap required, opened a new log
I20260805 22:46:21.078260 22057 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc833fc8dc74c8f82bc7318f882f738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } }
I20260805 22:46:21.078424 22057 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:21.078450 22057 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbc833fc8dc74c8f82bc7318f882f738, State: Initialized, Role: FOLLOWER
I20260805 22:46:21.078640 22057 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc833fc8dc74c8f82bc7318f882f738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } }
I20260805 22:46:21.078713 22057 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:21.078759 22057 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:21.078814 22057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:21.079411 22057 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc833fc8dc74c8f82bc7318f882f738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } }
I20260805 22:46:21.079509 22057 leader_election.cc:302] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbc833fc8dc74c8f82bc7318f882f738; no voters:
I20260805 22:46:21.079741 22057 leader_election.cc:288] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:21.079816 22060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:21.079949 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 21964
I20260805 22:46:21.080001 22060 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [term 1 LEADER]: Becoming Leader. State: Replica: fbc833fc8dc74c8f82bc7318f882f738, State: Running, Role: LEADER
I20260805 22:46:21.080051 22057 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:21.080099 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:21.080114 22060 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc833fc8dc74c8f82bc7318f882f738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } }
I20260805 22:46:21.080399 24341 external_mini_cluster.cc:1428] Setting key fd0817b70433695508d3f1e4a7c88e58
I20260805 22:46:21.080677 22062 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbc833fc8dc74c8f82bc7318f882f738" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc833fc8dc74c8f82bc7318f882f738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } } }
I20260805 22:46:21.080799 22062 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:21.080976 22064 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbc833fc8dc74c8f82bc7318f882f738. Latest consensus state: current_term: 1 leader_uuid: "fbc833fc8dc74c8f82bc7318f882f738" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc833fc8dc74c8f82bc7318f882f738" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } } }
I20260805 22:46:21.081082 22064 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:21.081614 22068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:21.082209 22068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:21.084059 22068 catalog_manager.cc:1347] Generated new cluster ID: 7c63ef6fb8544270ae6215cb27115860
I20260805 22:46:21.084113 22068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:46:21 dist-test-slave-g170 krb5kdc[21943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 22:46:21.093885 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:21.081284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54507 (local address 127.0.0.1:43637)
0805 22:46:21.081495 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:21.081506 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:21.081545 (+ 39us) server_negotiation.cc:407] Connection header received
0805 22:46:21.081674 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:21.081683 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:21.081755 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:21.081917 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:21.082913 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:21.084292 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:21.085503 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:21.086176 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:21.086294 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:21.089512 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:21.089517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:21.089517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:21.089531 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:21.091888 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:21.092391 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:21.092396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:21.092396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:21.092538 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:21.092838 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:21.092839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:21.092839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:21.093197 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:21.093397 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:21.093502 (+ 105us) server_negotiation.cc:299] Negotiation successful
0805 22:46:21.093633 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":114,"threads_started":1}
I20260805 22:46:21.094987 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:21.103506 22068 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:21.104087 22068 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:21.113690 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:21.103279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58505 (local address 127.0.0.1:43637)
0805 22:46:21.103334 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:21.103336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:21.103539 (+ 203us) server_negotiation.cc:407] Connection header received
0805 22:46:21.104939 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:21.104941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:21.104954 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:21.105033 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:21.105736 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:21.107131 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:21.108246 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:21.108624 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:21.108702 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:21.110431 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:21.110433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:21.110433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:21.110441 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:21.112297 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:21.112794 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:21.112795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:21.112795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:21.112862 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:21.113115 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:21.113116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:21.113116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:21.113257 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:21.113386 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:21.113503 (+ 117us) server_negotiation.cc:299] Negotiation successful
0805 22:46:21.113577 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260805 22:46:21.118407 22068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbc833fc8dc74c8f82bc7318f882f738: Generated new TSK 0
I20260805 22:46:21.118633 22068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:21.121934 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 21964
W20260805 22:46:21.133921 21941 connection.cc:535] client connection to 127.0.0.1:43637 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43637): Connection reset by peer (error 104)
2026-08-05T22:46:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1585 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:21 dist-test-slave-g170 krb5kdc[22099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:21 dist-test-slave-g170 krb5kdc[22099](info): set up 2 sockets
Aug 05 22:46:21 dist-test-slave-g170 krb5kdc[22099](info): commencing operation
krb5kdc: starting...
W20260805 22:46:22.387435 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:22 dist-test-slave-g170 krb5kdc[22099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:22.521401 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:22.605350 22117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:22.605597 22117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:22.605634 22117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:22.606779 22117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:22.606849 22117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:22.606874 22117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:22.606894 22117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:22.606925 22117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:22.608408 22117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:22.608696 22117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:22.608991 22117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:22.613962 22127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:22.613993 22117 server_base.cc:1034] running on GCE node
W20260805 22:46:22.614019 22128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:22.613976 22130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:22.614362 22117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:22.614663 22117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:22.615846 22117 hybrid_clock.cc:648] HybridClock initialized: now 1785969982615832 us; error 45 us; skew 500 ppm
Aug 05 22:46:22 dist-test-slave-g170 krb5kdc[22099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:22.618396 22117 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:22.618909 22117 webserver.cc:460] Webserver started at http://127.0.0.1:34337/ using document root <none> and password file <none>
I20260805 22:46:22.619135 22117 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:22.619203 22117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:22.619364 22117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:22.620427 22117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "5a0f8a21090a43f997402fe40ca9940e"
format_stamp: "Formatted at 2026-08-05 22:46:22 on dist-test-slave-g170"
server_key: "e509342617bb1043dd4da7162f7dcc5d"
server_key_iv: "e302f9bdded097094682dd3962a2e1cc"
server_key_version: "encryptionkey@0"
I20260805 22:46:22.620870 22117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "5a0f8a21090a43f997402fe40ca9940e"
format_stamp: "Formatted at 2026-08-05 22:46:22 on dist-test-slave-g170"
server_key: "e509342617bb1043dd4da7162f7dcc5d"
server_key_iv: "e302f9bdded097094682dd3962a2e1cc"
server_key_version: "encryptionkey@0"
I20260805 22:46:22.622248 22117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:22.623031 22138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:22.623227 22117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:22.623330 22117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "5a0f8a21090a43f997402fe40ca9940e"
format_stamp: "Formatted at 2026-08-05 22:46:22 on dist-test-slave-g170"
server_key: "e509342617bb1043dd4da7162f7dcc5d"
server_key_iv: "e302f9bdded097094682dd3962a2e1cc"
server_key_version: "encryptionkey@0"
I20260805 22:46:22.623416 22117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:22.650892 22117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:22.653164 22117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:22.653345 22117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:22.657862 22117 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40233
I20260805 22:46:22.657923 22210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40233 every 8 connection(s)
I20260805 22:46:22.658313 22117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:22.659267 22211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:22.661695 22211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e: Bootstrap starting.
I20260805 22:46:22.662312 22211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:22.662647 22211 log.cc:824] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:22.663354 22211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e: No bootstrap required, opened a new log
I20260805 22:46:22.664975 22211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f8a21090a43f997402fe40ca9940e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } }
I20260805 22:46:22.665109 22211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:22.665133 22211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0f8a21090a43f997402fe40ca9940e, State: Initialized, Role: FOLLOWER
I20260805 22:46:22.665247 22211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f8a21090a43f997402fe40ca9940e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } }
I20260805 22:46:22.665297 22211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:22.665318 22211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:22.665350 22211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:22.665839 22211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f8a21090a43f997402fe40ca9940e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } }
I20260805 22:46:22.665932 22211 leader_election.cc:302] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a0f8a21090a43f997402fe40ca9940e; no voters:
I20260805 22:46:22.666049 22211 leader_election.cc:288] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:22.666126 22214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:22.666322 22214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [term 1 LEADER]: Becoming Leader. State: Replica: 5a0f8a21090a43f997402fe40ca9940e, State: Running, Role: LEADER
I20260805 22:46:22.666347 22211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:22.666455 22214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f8a21090a43f997402fe40ca9940e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } }
I20260805 22:46:22.666977 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 22117
I20260805 22:46:22.667122 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:22.667308 22217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a0f8a21090a43f997402fe40ca9940e. Latest consensus state: current_term: 1 leader_uuid: "5a0f8a21090a43f997402fe40ca9940e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f8a21090a43f997402fe40ca9940e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } } }
I20260805 22:46:22.667402 22217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:22.667418 24341 external_mini_cluster.cc:1428] Setting key cf231e0c3d913a69f7678d3c0557e677
I20260805 22:46:22.667593 22215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a0f8a21090a43f997402fe40ca9940e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f8a21090a43f997402fe40ca9940e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } } }
I20260805 22:46:22.667691 22215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e [sys.catalog]: This master's current role is: LEADER
W20260805 22:46:22.668448 22235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:22.668541 22235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:22.668597 22230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:22.668932 22230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:22.670509 22230 catalog_manager.cc:1347] Generated new cluster ID: de60fb5495d94c259e93e921f2490351
I20260805 22:46:22.670567 22230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:22.674010 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:22.668016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52025 (local address 127.0.0.1:40233)
0805 22:46:22.668209 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:22.668217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:22.668272 (+ 55us) server_negotiation.cc:407] Connection header received
0805 22:46:22.668396 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:22.668403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:22.668462 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:22.668904 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:22.669020 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:22.670588 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:22.671703 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:22.672382 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:22.672512 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:22.673295 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:22.673299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:22.673299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:22.673511 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:22.673550 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:22.673565 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:22.673669 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:22.673748 (+ 79us) server_negotiation.cc:299] Negotiation successful
0805 22:46:22.673777 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260805 22:46:22.674988 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:22.680029 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:22.676474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36115 (local address 127.0.0.1:40233)
0805 22:46:22.676512 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:22.676514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:22.676564 (+ 50us) server_negotiation.cc:407] Connection header received
0805 22:46:22.676650 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:22.676651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:22.676662 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:22.676717 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:22.676881 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:22.677652 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:22.678664 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:22.678967 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:22.679043 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:22.679762 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:22.679764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:22.679765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:22.679770 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:22.679787 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:22.679791 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:22.679864 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:22.679936 (+ 72us) server_negotiation.cc:299] Negotiation successful
0805 22:46:22.679948 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:46:22.694402 22230 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:22.695247 22230 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:22.698386 22230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a0f8a21090a43f997402fe40ca9940e: Generated new TSK 0
I20260805 22:46:22.698575 22230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:22.701547 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 22117
W20260805 22:46:22.717629 22097 connection.cc:535] client connection to 127.0.0.1:40233 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40233): Connection reset by peer (error 104)
2026-08-05T22:46:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:22 dist-test-slave-g170 krb5kdc[22251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:22 dist-test-slave-g170 krb5kdc[22251](info): set up 2 sockets
Aug 05 22:46:22 dist-test-slave-g170 krb5kdc[22251](info): commencing operation
krb5kdc: starting...
W20260805 22:46:23.964004 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:24 dist-test-slave-g170 krb5kdc[22251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:24.095700 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:24.179407 22268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:24.179621 22268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:24.179646 22268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:24.180945 22268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:24.180987 22268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:24.181002 22268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:24.181015 22268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:24.181027 22268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:24.182509 22268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:24.182775 22268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:24.183022 22268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:24.187958 22276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:24.188015 22279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:24.188012 22277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:24.188150 22268 server_base.cc:1034] running on GCE node
I20260805 22:46:24.188448 22268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:24.188750 22268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:24.189955 22268 hybrid_clock.cc:648] HybridClock initialized: now 1785969984189941 us; error 45 us; skew 500 ppm
Aug 05 22:46:24 dist-test-slave-g170 krb5kdc[22251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:24.195801 22268 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:24.196285 22268 webserver.cc:460] Webserver started at http://127.0.0.1:41171/ using document root <none> and password file <none>
I20260805 22:46:24.196520 22268 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:24.196593 22268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:24.196753 22268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:24.197880 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "4da6e18626c64546b9d58f90529c0827"
format_stamp: "Formatted at 2026-08-05 22:46:24 on dist-test-slave-g170"
server_key: "26d816ae24140d4202226a1a66f27795"
server_key_iv: "7f725689938e73cde48a0268f14d837e"
server_key_version: "encryptionkey@0"
I20260805 22:46:24.198261 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "4da6e18626c64546b9d58f90529c0827"
format_stamp: "Formatted at 2026-08-05 22:46:24 on dist-test-slave-g170"
server_key: "26d816ae24140d4202226a1a66f27795"
server_key_iv: "7f725689938e73cde48a0268f14d837e"
server_key_version: "encryptionkey@0"
I20260805 22:46:24.199659 22268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:24.200449 22292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:24.200692 22268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:24.200820 22268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "4da6e18626c64546b9d58f90529c0827"
format_stamp: "Formatted at 2026-08-05 22:46:24 on dist-test-slave-g170"
server_key: "26d816ae24140d4202226a1a66f27795"
server_key_iv: "7f725689938e73cde48a0268f14d837e"
server_key_version: "encryptionkey@0"
I20260805 22:46:24.200893 22268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:24.204079 22268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:24.206049 22268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:24.206161 22268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:24.210134 22268 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42967
I20260805 22:46:24.210167 22369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42967 every 8 connection(s)
I20260805 22:46:24.210522 22268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:24.210969 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 22268
I20260805 22:46:24.211100 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:24.211294 24341 external_mini_cluster.cc:1428] Setting key 0cf23c840e3e2768280840304cd85dbf
I20260805 22:46:24.211856 22371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:24.213938 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:24.211870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45907 (local address 127.0.0.1:42967)
0805 22:46:24.212060 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:24.212074 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:24.212101 (+ 27us) server_negotiation.cc:407] Connection header received
0805 22:46:24.212550 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:24.212556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:24.212580 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:24.212733 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:24.213484 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:24.213488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:24.213488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:24.213492 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:24.213540 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:24.213551 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:24.213631 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:24.213702 (+ 71us) server_negotiation.cc:299] Negotiation successful
0805 22:46:24.213747 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260805 22:46:24.214987 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:24.215346 22371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827: Bootstrap starting.
I20260805 22:46:24.215996 22371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:24.216332 22371 log.cc:824] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:24.217152 22371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827: No bootstrap required, opened a new log
I20260805 22:46:24.217535 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:24.216440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39777 (local address 127.0.0.1:42967)
0805 22:46:24.216485 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:24.216487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:24.216617 (+ 130us) server_negotiation.cc:407] Connection header received
0805 22:46:24.216684 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:24.216686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:24.216690 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:24.216740 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:24.217321 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:24.217322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:24.217323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:24.217327 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:24.217340 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:24.217343 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:24.217399 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:24.217456 (+ 57us) server_negotiation.cc:299] Negotiation successful
0805 22:46:24.217470 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260805 22:46:24.219344 22371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da6e18626c64546b9d58f90529c0827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } }
I20260805 22:46:24.219480 22371 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:24.219503 22371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da6e18626c64546b9d58f90529c0827, State: Initialized, Role: FOLLOWER
I20260805 22:46:24.219645 22371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da6e18626c64546b9d58f90529c0827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } }
I20260805 22:46:24.219739 22371 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:24.219786 22371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:24.219838 22371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:24.220381 22371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da6e18626c64546b9d58f90529c0827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } }
I20260805 22:46:24.220498 22371 leader_election.cc:302] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4da6e18626c64546b9d58f90529c0827; no voters:
I20260805 22:46:24.220662 22371 leader_election.cc:288] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:24.220700 22389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:24.220888 22389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [term 1 LEADER]: Becoming Leader. State: Replica: 4da6e18626c64546b9d58f90529c0827, State: Running, Role: LEADER
I20260805 22:46:24.220963 22371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:24.221050 22389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da6e18626c64546b9d58f90529c0827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } }
I20260805 22:46:24.221403 22389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4da6e18626c64546b9d58f90529c0827. Latest consensus state: current_term: 1 leader_uuid: "4da6e18626c64546b9d58f90529c0827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da6e18626c64546b9d58f90529c0827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } } }
I20260805 22:46:24.221421 22390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4da6e18626c64546b9d58f90529c0827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da6e18626c64546b9d58f90529c0827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42967 } } }
I20260805 22:46:24.221536 22389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:24.221599 22390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:24.222105 22401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:24.222625 22401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:24.224182 22401 catalog_manager.cc:1347] Generated new cluster ID: 1b7694a55ae943eb9dbaf3eb4365456a
I20260805 22:46:24.224248 22401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:24.234665 22401 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:24.234834 22401 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:24.252346 22401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4da6e18626c64546b9d58f90529c0827: Generated new TSK 0
I20260805 22:46:24.252596 22401 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:24.255492 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 22268
2026-08-05T22:46:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:24 dist-test-slave-g170 krb5kdc[22417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:24 dist-test-slave-g170 krb5kdc[22417](info): set up 2 sockets
Aug 05 22:46:24 dist-test-slave-g170 krb5kdc[22417](info): commencing operation
krb5kdc: starting...
W20260805 22:46:25.604314 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:25 dist-test-slave-g170 krb5kdc[22417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:25.757949 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35577
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:25.845935 22432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:25.846161 22432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:25.846186 22432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:25.847311 22432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:25.847352 22432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:25.847365 22432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:25.847378 22432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:25.847390 22432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:25.848841 22432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:25.849031 22432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:25.849205 22432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:25.854336 22441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:25.854466 22432 server_base.cc:1034] running on GCE node
W20260805 22:46:25.854331 22443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:25.854331 22440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:25.854818 22432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:25.855116 22432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:25.856277 22432 hybrid_clock.cc:648] HybridClock initialized: now 1785969985856265 us; error 27 us; skew 500 ppm
Aug 05 22:46:25 dist-test-slave-g170 krb5kdc[22417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:25.859091 22432 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:25.859601 22432 webserver.cc:460] Webserver started at http://127.0.0.1:34323/ using document root <none> and password file <none>
I20260805 22:46:25.859839 22432 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:25.859911 22432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:25.860075 22432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:25.861294 22432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "da8dccb76ff640998fdd493f5025f967"
format_stamp: "Formatted at 2026-08-05 22:46:25 on dist-test-slave-g170"
server_key: "b4bd8493aa71e659affd22572a071557"
server_key_iv: "0266a7412473517e414e85bf467cf1b8"
server_key_version: "encryptionkey@0"
I20260805 22:46:25.861667 22432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "da8dccb76ff640998fdd493f5025f967"
format_stamp: "Formatted at 2026-08-05 22:46:25 on dist-test-slave-g170"
server_key: "b4bd8493aa71e659affd22572a071557"
server_key_iv: "0266a7412473517e414e85bf467cf1b8"
server_key_version: "encryptionkey@0"
I20260805 22:46:25.863052 22432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:46:25.863806 22452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:25.864022 22432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:25.864111 22432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "da8dccb76ff640998fdd493f5025f967"
format_stamp: "Formatted at 2026-08-05 22:46:25 on dist-test-slave-g170"
server_key: "b4bd8493aa71e659affd22572a071557"
server_key_iv: "0266a7412473517e414e85bf467cf1b8"
server_key_version: "encryptionkey@0"
I20260805 22:46:25.864212 22432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:25.882709 22432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:25.885128 22432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:25.885313 22432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:25.889783 22432 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35577
I20260805 22:46:25.889881 22531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35577 every 8 connection(s)
I20260805 22:46:25.890265 22432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:25.891268 22532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:25.893756 22532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967: Bootstrap starting.
I20260805 22:46:25.894340 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 22432
I20260805 22:46:25.894404 22532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:25.894439 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:25.894642 24341 external_mini_cluster.cc:1428] Setting key 9e97aeb9805bcc7385d7087d002d3f7d
I20260805 22:46:25.894707 22532 log.cc:824] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:25.895594 22532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967: No bootstrap required, opened a new log
I20260805 22:46:25.897948 22532 raft_consensus.cc:348] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8dccb76ff640998fdd493f5025f967" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35577 } }
I20260805 22:46:25.898154 22532 raft_consensus.cc:374] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:25.898192 22532 raft_consensus.cc:729] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8dccb76ff640998fdd493f5025f967, State: Initialized, Role: FOLLOWER
I20260805 22:46:25.898351 22532 consensus_queue.cc:260] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8dccb76ff640998fdd493f5025f967" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35577 } }
I20260805 22:46:25.898438 22532 raft_consensus.cc:388] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:25.898474 22532 raft_consensus.cc:482] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:25.898520 22532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:25.899245 22532 raft_consensus.cc:504] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8dccb76ff640998fdd493f5025f967" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35577 } }
I20260805 22:46:25.899369 22532 leader_election.cc:302] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da8dccb76ff640998fdd493f5025f967; no voters:
I20260805 22:46:25.899570 22532 leader_election.cc:288] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:25.899621 22539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:25.899855 22532 sys_catalog.cc:564] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:25.899924 22539 raft_consensus.cc:686] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [term 1 LEADER]: Becoming Leader. State: Replica: da8dccb76ff640998fdd493f5025f967, State: Running, Role: LEADER
I20260805 22:46:25.900121 22539 consensus_queue.cc:237] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8dccb76ff640998fdd493f5025f967" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35577 } }
I20260805 22:46:25.900529 22541 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da8dccb76ff640998fdd493f5025f967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8dccb76ff640998fdd493f5025f967" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35577 } } }
I20260805 22:46:25.900612 22541 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:25.900926 22553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:25.901480 22553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:25.901923 22540 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da8dccb76ff640998fdd493f5025f967. Latest consensus state: current_term: 1 leader_uuid: "da8dccb76ff640998fdd493f5025f967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da8dccb76ff640998fdd493f5025f967" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35577 } } }
I20260805 22:46:25.902025 22540 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967 [sys.catalog]: This master's current role is: LEADER
Aug 05 22:46:25 dist-test-slave-g170 krb5kdc[22417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 22:46:25.903155 22553 catalog_manager.cc:1347] Generated new cluster ID: 045f6fb9021c4243bdae3a226a39715b
I20260805 22:46:25.903218 22553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:25.906558 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:25.895158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48843 (local address 127.0.0.1:35577)
0805 22:46:25.895343 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:25.895352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:25.895378 (+ 26us) server_negotiation.cc:407] Connection header received
0805 22:46:25.895896 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:25.895904 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:25.895954 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:25.896073 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:25.896980 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:25.897908 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:25.899029 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:25.899618 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:25.899739 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:25.902921 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:25.902924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:25.902925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:25.902937 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:25.904900 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:25.905324 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:25.905328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:25.905328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:25.905430 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:25.905656 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:25.905657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:25.905657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:25.905944 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:25.906082 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:25.906230 (+ 148us) server_negotiation.cc:299] Negotiation successful
0805 22:46:25.906360 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260805 22:46:25.907743 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:25.916558 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:25.909162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43405 (local address 127.0.0.1:35577)
0805 22:46:25.909200 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:25.909201 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:25.909376 (+ 175us) server_negotiation.cc:407] Connection header received
0805 22:46:25.909408 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:25.909408 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:25.909419 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:25.909487 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:25.910064 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:25.910857 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:25.911859 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:25.912198 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:25.912274 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:25.913637 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:25.913639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:25.913639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:25.913648 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:25.915345 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:25.915741 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:25.915743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:25.915743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:25.915816 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:25.916018 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:25.916019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:25.916019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:25.916155 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:25.916258 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:25.916368 (+ 110us) server_negotiation.cc:299] Negotiation successful
0805 22:46:25.916448 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:46:25.925760 22553 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:25.926371 22553 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:25.933998 22553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da8dccb76ff640998fdd493f5025f967: Generated new TSK 0
I20260805 22:46:25.934219 22553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:25.939889 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 22432
2026-08-05T22:46:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1689 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:26 dist-test-slave-g170 krb5kdc[22575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:26 dist-test-slave-g170 krb5kdc[22575](info): set up 2 sockets
Aug 05 22:46:26 dist-test-slave-g170 krb5kdc[22575](info): commencing operation
krb5kdc: starting...
W20260805 22:46:27.273149 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:27 dist-test-slave-g170 krb5kdc[22575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:27.403740 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:27.492627 22595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:27.492895 22595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:27.492925 22595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:27.494153 22595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:27.494200 22595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:27.494215 22595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:27.494227 22595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:27.494239 22595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:27.495774 22595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:27.496042 22595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:27.496291 22595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:27.501806 22602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:27.501830 22605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:27.501883 22603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:27.502066 22595 server_base.cc:1034] running on GCE node
I20260805 22:46:27.502427 22595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:27.502694 22595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:27.503846 22595 hybrid_clock.cc:648] HybridClock initialized: now 1785969987503836 us; error 33 us; skew 500 ppm
Aug 05 22:46:27 dist-test-slave-g170 krb5kdc[22575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969987, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:27.506471 22595 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:27.506935 22595 webserver.cc:460] Webserver started at http://127.0.0.1:36285/ using document root <none> and password file <none>
I20260805 22:46:27.507165 22595 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:27.507242 22595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:27.507400 22595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:27.508477 22595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a"
format_stamp: "Formatted at 2026-08-05 22:46:27 on dist-test-slave-g170"
server_key: "b43761e28fb3d3aca6367bc7615488e4"
server_key_iv: "9e91236329db8649fa5e471e82188b2d"
server_key_version: "encryptionkey@0"
I20260805 22:46:27.508908 22595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a"
format_stamp: "Formatted at 2026-08-05 22:46:27 on dist-test-slave-g170"
server_key: "b43761e28fb3d3aca6367bc7615488e4"
server_key_iv: "9e91236329db8649fa5e471e82188b2d"
server_key_version: "encryptionkey@0"
I20260805 22:46:27.510612 22595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:27.511425 22613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:27.511611 22595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:46:27.511715 22595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a"
format_stamp: "Formatted at 2026-08-05 22:46:27 on dist-test-slave-g170"
server_key: "b43761e28fb3d3aca6367bc7615488e4"
server_key_iv: "9e91236329db8649fa5e471e82188b2d"
server_key_version: "encryptionkey@0"
I20260805 22:46:27.511811 22595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:27.527660 22595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:27.530062 22595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:27.530241 22595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:27.534449 22595 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42037
I20260805 22:46:27.534518 22692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42037 every 8 connection(s)
I20260805 22:46:27.534896 22595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:27.535904 22694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:27.538363 22694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a: Bootstrap starting.
I20260805 22:46:27.538972 22694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:27.539278 22694 log.cc:824] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:27.540118 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 22595
I20260805 22:46:27.540138 22694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a: No bootstrap required, opened a new log
I20260805 22:46:27.540263 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:27.540477 24341 external_mini_cluster.cc:1428] Setting key 9e1d4bc8a599f9868c1c51ed4b7ea2ce
I20260805 22:46:27.542232 22694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } }
I20260805 22:46:27.542371 22694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:27.542394 22694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4df0e2943e4b9ba969ccf60afbcd1a, State: Initialized, Role: FOLLOWER
I20260805 22:46:27.542532 22694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } }
I20260805 22:46:27.542620 22694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:27.542658 22694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:27.542779 22694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:27.543391 22694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } }
I20260805 22:46:27.543515 22694 leader_election.cc:302] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e4df0e2943e4b9ba969ccf60afbcd1a; no voters:
I20260805 22:46:27.543731 22694 leader_election.cc:288] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:27.543771 22700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:27.543957 22700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [term 1 LEADER]: Becoming Leader. State: Replica: 1e4df0e2943e4b9ba969ccf60afbcd1a, State: Running, Role: LEADER
I20260805 22:46:27.544085 22694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:27.544100 22700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } }
I20260805 22:46:27.544556 22700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e4df0e2943e4b9ba969ccf60afbcd1a. Latest consensus state: current_term: 1 leader_uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } } }
I20260805 22:46:27.544575 22702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4df0e2943e4b9ba969ccf60afbcd1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42037 } } }
I20260805 22:46:27.544718 22700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:27.544836 22702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:27.545125 22710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:27.545919 22710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:27.547859 22710 catalog_manager.cc:1347] Generated new cluster ID: d777e2c685a247eaa39d7b828f485596
I20260805 22:46:27.547873 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:27.541082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49257 (local address 127.0.0.1:42037)
0805 22:46:27.541316 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:27.541327 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:27.541360 (+ 33us) server_negotiation.cc:407] Connection header received
0805 22:46:27.542220 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:27.542229 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:27.542286 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:27.542485 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:27.542747 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:27.544268 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:27.545549 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:27.546143 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:27.546354 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:27.547088 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:27.547092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:27.547092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:27.547337 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:27.547376 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:27.547392 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:27.547497 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:27.547595 (+ 98us) server_negotiation.cc:299] Negotiation successful
0805 22:46:27.547660 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":104,"threads_started":1}
I20260805 22:46:27.547920 22710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:27.548978 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:27.553929 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:27.550472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35395 (local address 127.0.0.1:42037)
0805 22:46:27.550506 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:27.550508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:27.550645 (+ 137us) server_negotiation.cc:407] Connection header received
0805 22:46:27.550672 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:27.550673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:27.550684 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:27.550752 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:27.550882 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:27.551607 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:27.552578 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:27.552918 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:27.552987 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:27.553676 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:27.553677 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:27.553678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:27.553682 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:27.553701 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:27.553705 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:27.553769 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:27.553826 (+ 57us) server_negotiation.cc:299] Negotiation successful
0805 22:46:27.553850 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260805 22:46:27.562443 22710 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:27.563199 22710 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:27.567653 22710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e4df0e2943e4b9ba969ccf60afbcd1a: Generated new TSK 0
I20260805 22:46:27.567860 22710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:27.571215 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 22595
2026-08-05T22:46:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1630 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:27 dist-test-slave-g170 krb5kdc[22735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:27 dist-test-slave-g170 krb5kdc[22735](info): set up 2 sockets
Aug 05 22:46:27 dist-test-slave-g170 krb5kdc[22735](info): commencing operation
krb5kdc: starting...
W20260805 22:46:28.812999 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:28 dist-test-slave-g170 krb5kdc[22735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:28.937637 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:29.020601 22754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:29.020864 22754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:29.020891 22754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:29.022022 22754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:29.022060 22754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:29.022075 22754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:29.022087 22754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:29.022099 22754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:29.023630 22754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:29.023875 22754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:29.024143 22754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:29.029220 22765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:29.029223 22763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:29.029214 22760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:29.029246 22754 server_base.cc:1034] running on GCE node
I20260805 22:46:29.029680 22754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:29.029950 22754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:29.031127 22754 hybrid_clock.cc:648] HybridClock initialized: now 1785969989031093 us; error 55 us; skew 500 ppm
Aug 05 22:46:29 dist-test-slave-g170 krb5kdc[22735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:29.033901 22754 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:29.034390 22754 webserver.cc:460] Webserver started at http://127.0.0.1:34671/ using document root <none> and password file <none>
I20260805 22:46:29.034610 22754 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:29.034662 22754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:29.034755 22754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:29.035800 22754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "0893ba30d2a44720b575634d1de9b9ea"
format_stamp: "Formatted at 2026-08-05 22:46:29 on dist-test-slave-g170"
server_key: "c8562202d877029535f949c205777d83"
server_key_iv: "9ddcda9e2ed3245c8b7d1634c8b47f66"
server_key_version: "encryptionkey@0"
I20260805 22:46:29.036185 22754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "0893ba30d2a44720b575634d1de9b9ea"
format_stamp: "Formatted at 2026-08-05 22:46:29 on dist-test-slave-g170"
server_key: "c8562202d877029535f949c205777d83"
server_key_iv: "9ddcda9e2ed3245c8b7d1634c8b47f66"
server_key_version: "encryptionkey@0"
I20260805 22:46:29.037674 22754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:29.038426 22776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:29.038606 22754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:29.038759 22754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "0893ba30d2a44720b575634d1de9b9ea"
format_stamp: "Formatted at 2026-08-05 22:46:29 on dist-test-slave-g170"
server_key: "c8562202d877029535f949c205777d83"
server_key_iv: "9ddcda9e2ed3245c8b7d1634c8b47f66"
server_key_version: "encryptionkey@0"
I20260805 22:46:29.038852 22754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:29.042315 22754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:29.044659 22754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:29.044858 22754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:29.049090 22754 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34905
I20260805 22:46:29.049122 22847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34905 every 8 connection(s)
I20260805 22:46:29.049458 22754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:29.050506 22850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:29.052830 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 22754
I20260805 22:46:29.052928 22850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea: Bootstrap starting.
I20260805 22:46:29.052965 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:29.053136 24341 external_mini_cluster.cc:1428] Setting key e27c0828f25d28bf1fd363e82f5d57a9
I20260805 22:46:29.053668 22850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:29.054028 22850 log.cc:824] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:29.054870 22850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea: No bootstrap required, opened a new log
I20260805 22:46:29.055863 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:29.053636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37085 (local address 127.0.0.1:34905)
0805 22:46:29.053861 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:29.053869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:29.053894 (+ 25us) server_negotiation.cc:407] Connection header received
0805 22:46:29.054535 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:29.054545 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:29.054580 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:29.054702 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:29.055407 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:29.055411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:29.055411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:29.055415 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:29.055452 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:29.055462 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:29.055533 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:29.055602 (+ 69us) server_negotiation.cc:299] Negotiation successful
0805 22:46:29.055656 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":77,"threads_started":1}
I20260805 22:46:29.056990 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:29.057554 22850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0893ba30d2a44720b575634d1de9b9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } }
I20260805 22:46:29.057687 22850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:29.057719 22850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0893ba30d2a44720b575634d1de9b9ea, State: Initialized, Role: FOLLOWER
I20260805 22:46:29.057859 22850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0893ba30d2a44720b575634d1de9b9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } }
I20260805 22:46:29.057971 22850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:29.058018 22850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:29.058068 22850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:29.058662 22850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0893ba30d2a44720b575634d1de9b9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } }
I20260805 22:46:29.058796 22850 leader_election.cc:302] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0893ba30d2a44720b575634d1de9b9ea; no voters:
I20260805 22:46:29.058965 22850 leader_election.cc:288] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:29.059007 22865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:29.059217 22865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [term 1 LEADER]: Becoming Leader. State: Replica: 0893ba30d2a44720b575634d1de9b9ea, State: Running, Role: LEADER
I20260805 22:46:29.059228 22850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:29.059324 22865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0893ba30d2a44720b575634d1de9b9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } }
I20260805 22:46:29.059774 22866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0893ba30d2a44720b575634d1de9b9ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0893ba30d2a44720b575634d1de9b9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } } }
I20260805 22:46:29.059896 22866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:29.059774 22867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0893ba30d2a44720b575634d1de9b9ea. Latest consensus state: current_term: 1 leader_uuid: "0893ba30d2a44720b575634d1de9b9ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0893ba30d2a44720b575634d1de9b9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } } }
I20260805 22:46:29.060040 22867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:29.060217 22878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:29.060710 22878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:29.062309 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:29.060860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40379 (local address 127.0.0.1:34905)
0805 22:46:29.060900 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:29.060902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:29.061317 (+ 415us) server_negotiation.cc:407] Connection header received
0805 22:46:29.061405 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:29.061406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:29.061413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:29.061486 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:29.062062 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:29.062063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:29.062063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:29.062068 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:29.062083 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:29.062088 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:29.062156 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:29.062238 (+ 82us) server_negotiation.cc:299] Negotiation successful
0805 22:46:29.062251 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260805 22:46:29.062609 22878 catalog_manager.cc:1347] Generated new cluster ID: 0be7ea2110804af180df988124f76e5d
I20260805 22:46:29.062693 22878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:29.079269 22878 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:29.079470 22878 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:29.087527 22878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0893ba30d2a44720b575634d1de9b9ea: Generated new TSK 0
I20260805 22:46:29.087734 22878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:29.094986 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 22754
2026-08-05T22:46:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:29 dist-test-slave-g170 krb5kdc[22896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:29 dist-test-slave-g170 krb5kdc[22896](info): set up 2 sockets
Aug 05 22:46:29 dist-test-slave-g170 krb5kdc[22896](info): commencing operation
krb5kdc: starting...
W20260805 22:46:30.390507 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:30 dist-test-slave-g170 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:30.524827 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:30.613212 22913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:30.613451 22913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:30.613477 22913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:30.614689 22913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:30.614727 22913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:30.614743 22913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:30.614758 22913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:30.614770 22913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:30.616330 22913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:30.616537 22913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:30.616737 22913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:30.622110 22922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:30.622160 22923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:30.622241 22913 server_base.cc:1034] running on GCE node
W20260805 22:46:30.622179 22926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:30.622552 22913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:30.622804 22913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:30.623971 22913 hybrid_clock.cc:648] HybridClock initialized: now 1785969990623945 us; error 45 us; skew 500 ppm
Aug 05 22:46:30 dist-test-slave-g170 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969990, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:30.631724 22913 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:30.632247 22913 webserver.cc:460] Webserver started at http://127.0.0.1:44337/ using document root <none> and password file <none>
I20260805 22:46:30.632465 22913 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:30.632511 22913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:30.632609 22913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:30.633808 22913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "6809b90ffe6f47878793d9287f0627fa"
format_stamp: "Formatted at 2026-08-05 22:46:30 on dist-test-slave-g170"
server_key: "dd0dae884a309003b1c21c9e86afee95"
server_key_iv: "041fbbac9429db7258fe10e783160801"
server_key_version: "encryptionkey@0"
I20260805 22:46:30.634150 22913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "6809b90ffe6f47878793d9287f0627fa"
format_stamp: "Formatted at 2026-08-05 22:46:30 on dist-test-slave-g170"
server_key: "dd0dae884a309003b1c21c9e86afee95"
server_key_iv: "041fbbac9429db7258fe10e783160801"
server_key_version: "encryptionkey@0"
I20260805 22:46:30.635555 22913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:46:30.636328 22934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:30.636529 22913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:30.636621 22913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "6809b90ffe6f47878793d9287f0627fa"
format_stamp: "Formatted at 2026-08-05 22:46:30 on dist-test-slave-g170"
server_key: "dd0dae884a309003b1c21c9e86afee95"
server_key_iv: "041fbbac9429db7258fe10e783160801"
server_key_version: "encryptionkey@0"
I20260805 22:46:30.636682 22913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:30.660039 22913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:30.662539 22913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:30.662662 22913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:30.666721 22913 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41739
I20260805 22:46:30.666806 23002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41739 every 8 connection(s)
I20260805 22:46:30.667162 22913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:30.668200 23004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:30.670591 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 22913
I20260805 22:46:30.670740 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:30.670915 24341 external_mini_cluster.cc:1428] Setting key f72784a2601aba299be836b4ac85c4bf
I20260805 22:46:30.671236 23004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa: Bootstrap starting.
I20260805 22:46:30.671988 23004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:30.672317 23004 log.cc:824] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:30.673287 23004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa: No bootstrap required, opened a new log
I20260805 22:46:30.675431 23004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6809b90ffe6f47878793d9287f0627fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } }
I20260805 22:46:30.675601 23004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:30.675628 23004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6809b90ffe6f47878793d9287f0627fa, State: Initialized, Role: FOLLOWER
I20260805 22:46:30.675774 23004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6809b90ffe6f47878793d9287f0627fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } }
I20260805 22:46:30.675835 23004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:30.675856 23004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:30.675923 23004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:30.676679 23004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6809b90ffe6f47878793d9287f0627fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } }
I20260805 22:46:30.676823 23004 leader_election.cc:302] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6809b90ffe6f47878793d9287f0627fa; no voters:
I20260805 22:46:30.677008 23004 leader_election.cc:288] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:30.677035 23014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:30.677261 23014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [term 1 LEADER]: Becoming Leader. State: Replica: 6809b90ffe6f47878793d9287f0627fa, State: Running, Role: LEADER
I20260805 22:46:30.677304 23004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:30.677456 23014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6809b90ffe6f47878793d9287f0627fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } }
Aug 05 22:46:30 dist-test-slave-g170 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 22:46:30.678619 23017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6809b90ffe6f47878793d9287f0627fa. Latest consensus state: current_term: 1 leader_uuid: "6809b90ffe6f47878793d9287f0627fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6809b90ffe6f47878793d9287f0627fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } } }
I20260805 22:46:30.678709 23017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:30.678918 23016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6809b90ffe6f47878793d9287f0627fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6809b90ffe6f47878793d9287f0627fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } } }
I20260805 22:46:30.679051 23016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa [sys.catalog]: This master's current role is: LEADER
W20260805 22:46:30.679442 23043 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:30.679533 23043 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:30.679776 23044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:30.679971 23044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:30.681682 23044 catalog_manager.cc:1347] Generated new cluster ID: 3ddef1da708e4c55bb4534f8322dbf22
I20260805 22:46:30.681735 23044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:30.683202 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:30.671557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:37837 (local address 127.0.0.1:41739)
0805 22:46:30.671781 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:30.671791 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:30.671826 (+ 35us) server_negotiation.cc:407] Connection header received
0805 22:46:30.672315 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:30.672323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:30.672401 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:30.672561 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:30.673410 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:30.674329 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:30.675489 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:30.676090 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:30.676181 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:30.679198 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:30.679202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:30.679203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:30.679217 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:30.681349 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:30.681857 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:30.681860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:30.681861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:30.681969 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:30.682255 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:30.682256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:30.682256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:30.682571 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:30.682728 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:30.682864 (+ 136us) server_negotiation.cc:299] Negotiation successful
0805 22:46:30.682977 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
I20260805 22:46:30.684389 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:30.691365 23044 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:30.691962 23044 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:30.693576 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:30.685995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:42729 (local address 127.0.0.1:41739)
0805 22:46:30.686045 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:30.686047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:30.686195 (+ 148us) server_negotiation.cc:407] Connection header received
0805 22:46:30.686236 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:30.686237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:30.686248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:30.686322 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:30.686890 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:30.687685 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:30.688646 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:30.688987 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:30.689066 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:30.690482 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:30.690483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:30.690484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:30.690491 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:30.692307 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:30.692716 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:30.692717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:30.692717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:30.692822 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:30.693062 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:30.693063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:30.693064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:30.693191 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:30.693305 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:30.693421 (+ 116us) server_negotiation.cc:299] Negotiation successful
0805 22:46:30.693488 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260805 22:46:30.700865 23044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6809b90ffe6f47878793d9287f0627fa: Generated new TSK 0
I20260805 22:46:30.701061 23044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:30.705937 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 22913
W20260805 22:46:30.717788 22893 connection.cc:535] client connection to 127.0.0.1:41739 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41739): Connection reset by peer (error 104)
2026-08-05T22:46:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:30 dist-test-slave-g170 krb5kdc[23058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:30 dist-test-slave-g170 krb5kdc[23058](info): set up 2 sockets
Aug 05 22:46:30 dist-test-slave-g170 krb5kdc[23058](info): commencing operation
krb5kdc: starting...
W20260805 22:46:31.948369 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:32 dist-test-slave-g170 krb5kdc[23058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:32.077840 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:32.162241 23082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:32.162497 23082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:32.162534 23082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:32.163712 23082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:32.163784 23082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:32.163807 23082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:32.163829 23082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:32.163859 23082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:32.165405 23082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:32.165695 23082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:32.165928 23082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:32.171017 23089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:32.171094 23090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:32.171320 23082 server_base.cc:1034] running on GCE node
W20260805 22:46:32.171021 23093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:32.171667 23082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:32.172022 23082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:32.173187 23082 hybrid_clock.cc:648] HybridClock initialized: now 1785969992173159 us; error 43 us; skew 500 ppm
Aug 05 22:46:32 dist-test-slave-g170 krb5kdc[23058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969992, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:32.175890 23082 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:32.176285 23082 webserver.cc:460] Webserver started at http://127.0.0.1:38733/ using document root <none> and password file <none>
I20260805 22:46:32.176499 23082 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:32.176548 23082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:32.176662 23082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:32.177788 23082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "977cadb24ace460fbd0c192961951e71"
format_stamp: "Formatted at 2026-08-05 22:46:32 on dist-test-slave-g170"
server_key: "c26cc9671ff63d5cdbcdb13358f84275"
server_key_iv: "e673718784f3e1ae681db6e2d69252d5"
server_key_version: "encryptionkey@0"
I20260805 22:46:32.178104 23082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "977cadb24ace460fbd0c192961951e71"
format_stamp: "Formatted at 2026-08-05 22:46:32 on dist-test-slave-g170"
server_key: "c26cc9671ff63d5cdbcdb13358f84275"
server_key_iv: "e673718784f3e1ae681db6e2d69252d5"
server_key_version: "encryptionkey@0"
I20260805 22:46:32.179479 23082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:46:32.180281 23103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:32.180482 23082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:32.180568 23082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "977cadb24ace460fbd0c192961951e71"
format_stamp: "Formatted at 2026-08-05 22:46:32 on dist-test-slave-g170"
server_key: "c26cc9671ff63d5cdbcdb13358f84275"
server_key_iv: "e673718784f3e1ae681db6e2d69252d5"
server_key_version: "encryptionkey@0"
I20260805 22:46:32.180624 23082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:32.190390 23082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:32.192847 23082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:32.192968 23082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:32.197120 23082 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41323
I20260805 22:46:32.197187 23173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41323 every 8 connection(s)
I20260805 22:46:32.197538 23082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:32.198495 23174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:32.201125 23174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71: Bootstrap starting.
I20260805 22:46:32.201769 23174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:32.202109 23174 log.cc:824] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:32.202936 23174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71: No bootstrap required, opened a new log
I20260805 22:46:32.203706 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 23082
I20260805 22:46:32.203841 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:32.204085 24341 external_mini_cluster.cc:1428] Setting key e846e34d35dc1776f1e79b1972d2685f
I20260805 22:46:32.205358 23174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cadb24ace460fbd0c192961951e71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41323 } }
I20260805 22:46:32.205534 23174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:32.205606 23174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977cadb24ace460fbd0c192961951e71, State: Initialized, Role: FOLLOWER
I20260805 22:46:32.205806 23174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cadb24ace460fbd0c192961951e71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41323 } }
I20260805 22:46:32.205910 23174 raft_consensus.cc:388] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:32.205951 23174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:32.206013 23174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:32.206873 23174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cadb24ace460fbd0c192961951e71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41323 } }
I20260805 22:46:32.206992 23174 leader_election.cc:302] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 977cadb24ace460fbd0c192961951e71; no voters:
I20260805 22:46:32.207178 23174 leader_election.cc:288] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:32.207240 23183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:32.207428 23183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [term 1 LEADER]: Becoming Leader. State: Replica: 977cadb24ace460fbd0c192961951e71, State: Running, Role: LEADER
I20260805 22:46:32.207499 23174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:32.207571 23183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cadb24ace460fbd0c192961951e71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41323 } }
I20260805 22:46:32.207916 23183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 977cadb24ace460fbd0c192961951e71. Latest consensus state: current_term: 1 leader_uuid: "977cadb24ace460fbd0c192961951e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cadb24ace460fbd0c192961951e71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41323 } } }
I20260805 22:46:32.208109 23183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:32.208050 23184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "977cadb24ace460fbd0c192961951e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cadb24ace460fbd0c192961951e71" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41323 } } }
I20260805 22:46:32.208400 23184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:32.208448 23191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:32.209103 23191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:32.211397 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:32.204664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:47895 (local address 127.0.0.1:41323)
0805 22:46:32.204885 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:32.204896 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:32.204931 (+ 35us) server_negotiation.cc:407] Connection header received
0805 22:46:32.205339 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:32.205347 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:32.205400 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:32.205544 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:32.205855 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:32.207784 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:32.208913 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:32.209530 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:32.209643 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:32.210540 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:32.210544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:32.210545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:32.210800 (+ 255us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:32.210852 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:32.210868 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:32.210971 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:32.211128 (+ 157us) server_negotiation.cc:299] Negotiation successful
0805 22:46:32.211161 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
I20260805 22:46:32.211510 23191 catalog_manager.cc:1347] Generated new cluster ID: 07672fa9d78b480998c287ebd906c474
I20260805 22:46:32.211575 23191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:32.212497 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:32.217921 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:32.214237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:45029 (local address 127.0.0.1:41323)
0805 22:46:32.214275 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:32.214277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:32.214353 (+ 76us) server_negotiation.cc:407] Connection header received
0805 22:46:32.214409 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:32.214410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:32.214424 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:32.214501 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:32.214640 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:32.215459 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:32.216441 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:32.216851 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:32.216948 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:32.217643 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:32.217644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:32.217645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:32.217650 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:32.217666 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:32.217671 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:32.217736 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:32.217823 (+ 87us) server_negotiation.cc:299] Negotiation successful
0805 22:46:32.217837 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:46:32.222245 23191 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:32.222833 23191 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:32.226305 23191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 977cadb24ace460fbd0c192961951e71: Generated new TSK 0
I20260805 22:46:32.226488 23191 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:32.228930 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 23082
W20260805 22:46:32.242097 23055 connection.cc:535] client connection to 127.0.0.1:41323 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41323): Connection reset by peer (error 104)
2026-08-05T22:46:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:32 dist-test-slave-g170 krb5kdc[23216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:32 dist-test-slave-g170 krb5kdc[23216](info): set up 2 sockets
Aug 05 22:46:32 dist-test-slave-g170 krb5kdc[23216](info): commencing operation
krb5kdc: starting...
W20260805 22:46:33.528612 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:33 dist-test-slave-g170 krb5kdc[23216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:33.660496 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40987
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:33.746029 23237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:33.746322 23237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:33.746387 23237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:33.747710 23237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:33.747793 23237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:33.747822 23237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:33.747853 23237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:33.747881 23237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:33.749621 23237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:33.749931 23237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:33.750195 23237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:33.755591 23246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:33.755656 23245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:33.755587 23248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:33.755908 23237 server_base.cc:1034] running on GCE node
I20260805 22:46:33.756199 23237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:33.756688 23237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:33.757979 23237 hybrid_clock.cc:648] HybridClock initialized: now 1785969993757950 us; error 39 us; skew 500 ppm
Aug 05 22:46:33 dist-test-slave-g170 krb5kdc[23216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:33.760879 23237 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:33.761308 23237 webserver.cc:460] Webserver started at http://127.0.0.1:44685/ using document root <none> and password file <none>
I20260805 22:46:33.761546 23237 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:33.761595 23237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:33.761708 23237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:33.762802 23237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "0a118b6e305247908bb72b74a62ede6d"
format_stamp: "Formatted at 2026-08-05 22:46:33 on dist-test-slave-g170"
server_key: "1769075fed10bf410e996f2e96dbc6eb"
server_key_iv: "5dfd501cd954577eb0249ba59f574abe"
server_key_version: "encryptionkey@0"
I20260805 22:46:33.763195 23237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "0a118b6e305247908bb72b74a62ede6d"
format_stamp: "Formatted at 2026-08-05 22:46:33 on dist-test-slave-g170"
server_key: "1769075fed10bf410e996f2e96dbc6eb"
server_key_iv: "5dfd501cd954577eb0249ba59f574abe"
server_key_version: "encryptionkey@0"
I20260805 22:46:33.764642 23237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260805 22:46:33.765487 23258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:33.765807 23237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:33.765920 23237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "0a118b6e305247908bb72b74a62ede6d"
format_stamp: "Formatted at 2026-08-05 22:46:33 on dist-test-slave-g170"
server_key: "1769075fed10bf410e996f2e96dbc6eb"
server_key_iv: "5dfd501cd954577eb0249ba59f574abe"
server_key_version: "encryptionkey@0"
I20260805 22:46:33.765995 23237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:33.769366 23237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:33.771442 23237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:33.771598 23237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:33.775660 23237 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40987
I20260805 22:46:33.775714 23327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40987 every 8 connection(s)
I20260805 22:46:33.776083 23237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:33.776436 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 23237
I20260805 22:46:33.776574 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:33.776876 24341 external_mini_cluster.cc:1428] Setting key 3d432d75c73a956b24b34504bcf1ecc1
I20260805 22:46:33.777565 23328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:33.779477 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:33.777484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:32967 (local address 127.0.0.1:40987)
0805 22:46:33.777675 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:33.777698 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:33.777736 (+ 38us) server_negotiation.cc:407] Connection header received
0805 22:46:33.778178 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:33.778185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:33.778218 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:33.778345 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:33.779000 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:33.779003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:33.779004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:33.779007 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:33.779048 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:33.779059 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:33.779113 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:33.779224 (+ 111us) server_negotiation.cc:299] Negotiation successful
0805 22:46:33.779283 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260805 22:46:33.780603 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:33.781451 23328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d: Bootstrap starting.
I20260805 22:46:33.782397 23328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:33.782824 23328 log.cc:824] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:33.783766 23328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d: No bootstrap required, opened a new log
I20260805 22:46:33.784368 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:33.782497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:54561 (local address 127.0.0.1:40987)
0805 22:46:33.782546 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:33.782548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:33.783213 (+ 665us) server_negotiation.cc:407] Connection header received
0805 22:46:33.783292 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:33.783293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:33.783299 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:33.783363 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:33.784124 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:33.784125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:33.784125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:33.784130 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:33.784145 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:33.784149 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:33.784267 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:33.784285 (+ 18us) server_negotiation.cc:299] Negotiation successful
0805 22:46:33.784297 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260805 22:46:33.786221 23328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a118b6e305247908bb72b74a62ede6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40987 } }
I20260805 22:46:33.786371 23328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:33.786398 23328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a118b6e305247908bb72b74a62ede6d, State: Initialized, Role: FOLLOWER
I20260805 22:46:33.786545 23328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a118b6e305247908bb72b74a62ede6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40987 } }
I20260805 22:46:33.786630 23328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:33.786656 23328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:33.786727 23328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:33.787586 23328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a118b6e305247908bb72b74a62ede6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40987 } }
I20260805 22:46:33.787734 23328 leader_election.cc:302] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a118b6e305247908bb72b74a62ede6d; no voters:
I20260805 22:46:33.787937 23328 leader_election.cc:288] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:33.787974 23345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:33.788167 23345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [term 1 LEADER]: Becoming Leader. State: Replica: 0a118b6e305247908bb72b74a62ede6d, State: Running, Role: LEADER
I20260805 22:46:33.788255 23328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:33.788307 23345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a118b6e305247908bb72b74a62ede6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40987 } }
I20260805 22:46:33.788708 23350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a118b6e305247908bb72b74a62ede6d. Latest consensus state: current_term: 1 leader_uuid: "0a118b6e305247908bb72b74a62ede6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a118b6e305247908bb72b74a62ede6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40987 } } }
I20260805 22:46:33.788833 23350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:33.788697 23349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a118b6e305247908bb72b74a62ede6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a118b6e305247908bb72b74a62ede6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40987 } } }
I20260805 22:46:33.789083 23355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:33.789105 23349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:33.789968 23355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:33.791707 23355 catalog_manager.cc:1347] Generated new cluster ID: 44c0e4e077bd45b6a19b5514b52363ec
I20260805 22:46:33.791764 23355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:33.797315 23355 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:33.797501 23355 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:33.802510 23355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a118b6e305247908bb72b74a62ede6d: Generated new TSK 0
I20260805 22:46:33.802801 23355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:33.813647 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 23237
2026-08-05T22:46:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:33 dist-test-slave-g170 krb5kdc[23378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:33 dist-test-slave-g170 krb5kdc[23378](info): set up 2 sockets
Aug 05 22:46:33 dist-test-slave-g170 krb5kdc[23378](info): commencing operation
krb5kdc: starting...
W20260805 22:46:35.053695 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:35 dist-test-slave-g170 krb5kdc[23378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:35.191753 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:35.278349 23399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:35.278563 23399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:35.278586 23399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:35.279693 23399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:35.279738 23399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:35.279752 23399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:35.279764 23399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:35.279776 23399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:35.281224 23399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46365
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:35.281419 23399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:35.281585 23399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:35.286474 23412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:35.286547 23409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:35.286643 23399 server_base.cc:1034] running on GCE node
W20260805 22:46:35.286784 23410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:35.287060 23399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:35.287391 23399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:35.288549 23399 hybrid_clock.cc:648] HybridClock initialized: now 1785969995288536 us; error 34 us; skew 500 ppm
Aug 05 22:46:35 dist-test-slave-g170 krb5kdc[23378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:35.291478 23399 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:35.292040 23399 webserver.cc:460] Webserver started at http://127.0.0.1:41353/ using document root <none> and password file <none>
I20260805 22:46:35.292316 23399 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:35.292392 23399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:35.292547 23399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:35.293785 23399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "4c4f8c9e7b60464780487b98666c5a6e"
format_stamp: "Formatted at 2026-08-05 22:46:35 on dist-test-slave-g170"
server_key: "89a12aec0ee9dceb3ecc2556af5fe80f"
server_key_iv: "e5f90ae66ab2425e3e83cee9fc2ab745"
server_key_version: "encryptionkey@0"
I20260805 22:46:35.294202 23399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "4c4f8c9e7b60464780487b98666c5a6e"
format_stamp: "Formatted at 2026-08-05 22:46:35 on dist-test-slave-g170"
server_key: "89a12aec0ee9dceb3ecc2556af5fe80f"
server_key_iv: "e5f90ae66ab2425e3e83cee9fc2ab745"
server_key_version: "encryptionkey@0"
I20260805 22:46:35.295689 23399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:35.296516 23423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:35.296803 23399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:35.296895 23399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "4c4f8c9e7b60464780487b98666c5a6e"
format_stamp: "Formatted at 2026-08-05 22:46:35 on dist-test-slave-g170"
server_key: "89a12aec0ee9dceb3ecc2556af5fe80f"
server_key_iv: "e5f90ae66ab2425e3e83cee9fc2ab745"
server_key_version: "encryptionkey@0"
I20260805 22:46:35.296953 23399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:35.310988 23399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:35.313479 23399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:35.313607 23399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:35.317924 23399 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46365
I20260805 22:46:35.318001 23500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46365 every 8 connection(s)
I20260805 22:46:35.318380 23399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:35.319396 23501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:35.321995 23501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e: Bootstrap starting.
I20260805 22:46:35.322657 23501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:35.322971 23501 log.cc:824] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:35.323777 23501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e: No bootstrap required, opened a new log
I20260805 22:46:35.325479 23501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4f8c9e7b60464780487b98666c5a6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } }
I20260805 22:46:35.325649 23501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:35.325672 23501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c4f8c9e7b60464780487b98666c5a6e, State: Initialized, Role: FOLLOWER
I20260805 22:46:35.325836 23501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4f8c9e7b60464780487b98666c5a6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } }
I20260805 22:46:35.325901 23501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:35.325923 23501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:35.325971 23501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:35.326573 23501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4f8c9e7b60464780487b98666c5a6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } }
I20260805 22:46:35.326663 23501 leader_election.cc:302] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c4f8c9e7b60464780487b98666c5a6e; no voters:
I20260805 22:46:35.326844 23501 leader_election.cc:288] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:35.326893 23506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:35.327147 23501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:35.327152 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 23399
I20260805 22:46:35.327152 23506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [term 1 LEADER]: Becoming Leader. State: Replica: 4c4f8c9e7b60464780487b98666c5a6e, State: Running, Role: LEADER
I20260805 22:46:35.327266 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:35.327378 23506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4f8c9e7b60464780487b98666c5a6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } }
I20260805 22:46:35.327488 24341 external_mini_cluster.cc:1428] Setting key a38b00c624c3f6c114e60f7c8575c225
I20260805 22:46:35.328004 23509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c4f8c9e7b60464780487b98666c5a6e. Latest consensus state: current_term: 1 leader_uuid: "4c4f8c9e7b60464780487b98666c5a6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4f8c9e7b60464780487b98666c5a6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } } }
I20260805 22:46:35.328096 23509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:35.328809 23518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:35.328953 23508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c4f8c9e7b60464780487b98666c5a6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4f8c9e7b60464780487b98666c5a6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } } }
I20260805 22:46:35.329041 23508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:35.329456 23518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:35.331066 23518 catalog_manager.cc:1347] Generated new cluster ID: 0ceb0f42fe584184befae2b73d85befe
I20260805 22:46:35.331120 23518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:46:35 dist-test-slave-g170 krb5kdc[23378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 22:46:35.340242 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:35.328240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:57687 (local address 127.0.0.1:46365)
0805 22:46:35.328708 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:35.328717 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:35.328756 (+ 39us) server_negotiation.cc:407] Connection header received
0805 22:46:35.328926 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:35.328935 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:35.329034 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:35.330061 (+ 1027us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:35.330162 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:35.331479 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:35.332551 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:35.333230 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:35.333342 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:35.336311 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:35.336316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:35.336316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:35.336330 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:35.338372 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:35.338854 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:35.338857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:35.338858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:35.338972 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:35.339274 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:35.339276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:35.339276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:35.339588 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:35.339751 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:35.339908 (+ 157us) server_negotiation.cc:299] Negotiation successful
0805 22:46:35.340018 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":199,"threads_started":1}
I20260805 22:46:35.341323 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:35.350404 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:35.342828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:59871 (local address 127.0.0.1:46365)
0805 22:46:35.342867 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:35.342868 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:35.342983 (+ 115us) server_negotiation.cc:407] Connection header received
0805 22:46:35.343071 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:35.343072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:35.343082 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:35.343110 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:35.343803 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:35.344586 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:35.345677 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:35.345992 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:35.346031 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:35.347483 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:35.347484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:35.347484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:35.347491 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:35.349185 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:35.349600 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:35.349601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:35.349601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:35.349653 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:35.349881 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:35.349882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:35.349882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:35.350020 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:35.350129 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:35.350252 (+ 123us) server_negotiation.cc:299] Negotiation successful
0805 22:46:35.350316 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:46:35.356060 23518 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:35.356666 23518 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:35.360087 23518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c4f8c9e7b60464780487b98666c5a6e: Generated new TSK 0
I20260805 22:46:35.360275 23518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:35.363966 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 23399
W20260805 22:46:35.378532 23372 connection.cc:535] client connection to 127.0.0.1:46365 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46365): Connection reset by peer (error 104)
2026-08-05T22:46:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:35 dist-test-slave-g170 krb5kdc[23548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:35 dist-test-slave-g170 krb5kdc[23548](info): set up 2 sockets
Aug 05 22:46:35 dist-test-slave-g170 krb5kdc[23548](info): commencing operation
krb5kdc: starting...
W20260805 22:46:36.725009 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:36 dist-test-slave-g170 krb5kdc[23548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:36.855425 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36687
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:36.941429 23565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:36.941640 23565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:36.941664 23565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:36.942762 23565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:36.942798 23565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:36.942812 23565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:36.942824 23565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:36.942836 23565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:36.944314 23565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36687
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:36.944520 23565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:36.944711 23565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:36.949771 23574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:36.949771 23577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:36.949764 23572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:36.950050 23565 server_base.cc:1034] running on GCE node
I20260805 22:46:36.950311 23565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:36.950558 23565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:36.951732 23565 hybrid_clock.cc:648] HybridClock initialized: now 1785969996951712 us; error 27 us; skew 500 ppm
Aug 05 22:46:36 dist-test-slave-g170 krb5kdc[23548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969996, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:36.954491 23565 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:36.954948 23565 webserver.cc:460] Webserver started at http://127.0.0.1:39027/ using document root <none> and password file <none>
I20260805 22:46:36.955211 23565 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:36.955283 23565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:36.955435 23565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:36.956538 23565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "891e502680714abe87de9a4a64879819"
format_stamp: "Formatted at 2026-08-05 22:46:36 on dist-test-slave-g170"
server_key: "d0596c44bdb5dbf12d19e653ded34b31"
server_key_iv: "5b91212c22720c923a6e0bf0168659ae"
server_key_version: "encryptionkey@0"
I20260805 22:46:36.956933 23565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "891e502680714abe87de9a4a64879819"
format_stamp: "Formatted at 2026-08-05 22:46:36 on dist-test-slave-g170"
server_key: "d0596c44bdb5dbf12d19e653ded34b31"
server_key_iv: "5b91212c22720c923a6e0bf0168659ae"
server_key_version: "encryptionkey@0"
I20260805 22:46:36.958359 23565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:36.959276 23585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:36.959535 23565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:46:36.959650 23565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "891e502680714abe87de9a4a64879819"
format_stamp: "Formatted at 2026-08-05 22:46:36 on dist-test-slave-g170"
server_key: "d0596c44bdb5dbf12d19e653ded34b31"
server_key_iv: "5b91212c22720c923a6e0bf0168659ae"
server_key_version: "encryptionkey@0"
I20260805 22:46:36.959736 23565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:36.986689 23565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:36.989415 23565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:36.989626 23565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:36.994377 23565 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36687
I20260805 22:46:36.994418 23665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36687 every 8 connection(s)
I20260805 22:46:36.994813 23565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:36.995925 23666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:36.998567 23666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819: Bootstrap starting.
I20260805 22:46:36.999234 23666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:36.999557 23666 log.cc:824] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:37.000355 23666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819: No bootstrap required, opened a new log
I20260805 22:46:37.002053 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 23565
I20260805 22:46:37.002208 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:37.002476 24341 external_mini_cluster.cc:1428] Setting key fa73466e979ff1db0733cc79f4f9611b
I20260805 22:46:37.002753 23666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891e502680714abe87de9a4a64879819" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36687 } }
I20260805 22:46:37.002960 23666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:37.003010 23666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 891e502680714abe87de9a4a64879819, State: Initialized, Role: FOLLOWER
I20260805 22:46:37.003197 23666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891e502680714abe87de9a4a64879819" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36687 } }
I20260805 22:46:37.003334 23666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:37.003417 23666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:37.003479 23666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:37.004163 23666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891e502680714abe87de9a4a64879819" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36687 } }
I20260805 22:46:37.004292 23666 leader_election.cc:302] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 891e502680714abe87de9a4a64879819; no voters:
I20260805 22:46:37.004493 23666 leader_election.cc:288] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:37.004585 23675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:37.004724 23675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [term 1 LEADER]: Becoming Leader. State: Replica: 891e502680714abe87de9a4a64879819, State: Running, Role: LEADER
I20260805 22:46:37.004861 23666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:37.004923 23675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891e502680714abe87de9a4a64879819" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36687 } }
I20260805 22:46:37.005405 23675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 891e502680714abe87de9a4a64879819. Latest consensus state: current_term: 1 leader_uuid: "891e502680714abe87de9a4a64879819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891e502680714abe87de9a4a64879819" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36687 } } }
I20260805 22:46:37.005544 23675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:37.005693 23676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "891e502680714abe87de9a4a64879819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891e502680714abe87de9a4a64879819" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36687 } } }
I20260805 22:46:37.005810 23685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:37.005893 23676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:37.006560 23685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:37.009541 23673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:37.003104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:36979 (local address 127.0.0.1:36687)
0805 22:46:37.003346 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:37.003364 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:37.003394 (+ 30us) server_negotiation.cc:407] Connection header received
0805 22:46:37.003536 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:37.003546 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:37.003626 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:37.003825 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:37.004244 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:37.005882 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:37.007184 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:37.007669 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:37.007807 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:37.008684 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:37.008688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:37.008688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:37.008943 (+ 255us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:37.008989 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:37.009010 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:37.009122 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:37.009216 (+ 94us) server_negotiation.cc:299] Negotiation successful
0805 22:46:37.009249 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":121,"threads_started":1}
I20260805 22:46:37.009876 23685 catalog_manager.cc:1347] Generated new cluster ID: 9305919aa2ba48bf8c5490d92d418368
I20260805 22:46:37.009941 23685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:37.010581 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:37.015502 23673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:37.012094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:32773 (local address 127.0.0.1:36687)
0805 22:46:37.012154 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:37.012156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:37.012312 (+ 156us) server_negotiation.cc:407] Connection header received
0805 22:46:37.012391 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:37.012392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:37.012403 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:37.012455 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:37.012625 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:37.013264 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:37.014064 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:37.014344 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:37.014440 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:37.015208 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:37.015209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:37.015210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:37.015215 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:37.015230 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:37.015234 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:37.015298 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:37.015379 (+ 81us) server_negotiation.cc:299] Negotiation successful
0805 22:46:37.015393 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":23}
I20260805 22:46:37.035032 23685 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:37.035780 23685 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:37.041746 23685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 891e502680714abe87de9a4a64879819: Generated new TSK 0
I20260805 22:46:37.041965 23685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:37.046213 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 23565
W20260805 22:46:37.055713 23544 connection.cc:535] client connection to 127.0.0.1:36687 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36687): Connection reset by peer (error 104)
2026-08-05T22:46:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1676 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:37 dist-test-slave-g170 krb5kdc[23710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:37 dist-test-slave-g170 krb5kdc[23710](info): set up 2 sockets
Aug 05 22:46:37 dist-test-slave-g170 krb5kdc[23710](info): commencing operation
krb5kdc: starting...
W20260805 22:46:38.354297 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:38 dist-test-slave-g170 krb5kdc[23710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 22:46:38.488253 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:38.573832 23730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:38.574028 23730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:38.574050 23730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:38.575119 23730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:38.575155 23730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:38.575168 23730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:38.575181 23730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:38.575201 23730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:38.576582 23730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46261
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:38.576825 23730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:38.577026 23730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 22:46:38.581961 23730 server_base.cc:1034] running on GCE node
W20260805 22:46:38.581904 23738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:38.581904 23739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:38.581931 23744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:38.582443 23730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:38.582731 23730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:38.583886 23730 hybrid_clock.cc:648] HybridClock initialized: now 1785969998583867 us; error 39 us; skew 500 ppm
Aug 05 22:46:38 dist-test-slave-g170 krb5kdc[23710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:38.586633 23730 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 22:46:38.587059 23730 webserver.cc:460] Webserver started at http://127.0.0.1:45589/ using document root <none> and password file <none>
I20260805 22:46:38.587306 23730 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:38.587376 23730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:38.587515 23730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:38.588913 23730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "859dd0ec1a704046add1af1fac22aa0f"
format_stamp: "Formatted at 2026-08-05 22:46:38 on dist-test-slave-g170"
server_key: "455937629bdc9fcee599e370216f7b04"
server_key_iv: "2bfda83755bac7b856b8509207d0e1ba"
server_key_version: "encryptionkey@0"
I20260805 22:46:38.589335 23730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "859dd0ec1a704046add1af1fac22aa0f"
format_stamp: "Formatted at 2026-08-05 22:46:38 on dist-test-slave-g170"
server_key: "455937629bdc9fcee599e370216f7b04"
server_key_iv: "2bfda83755bac7b856b8509207d0e1ba"
server_key_version: "encryptionkey@0"
I20260805 22:46:38.590905 23730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:46:38.591857 23753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:38.592136 23730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:38.592269 23730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "859dd0ec1a704046add1af1fac22aa0f"
format_stamp: "Formatted at 2026-08-05 22:46:38 on dist-test-slave-g170"
server_key: "455937629bdc9fcee599e370216f7b04"
server_key_iv: "2bfda83755bac7b856b8509207d0e1ba"
server_key_version: "encryptionkey@0"
I20260805 22:46:38.592365 23730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:38.595731 23730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:38.598134 23730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:38.598304 23730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:38.602427 23730 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46261
I20260805 22:46:38.602463 23822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46261 every 8 connection(s)
I20260805 22:46:38.602825 23730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:38.603885 23825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:38.604061 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 23730
I20260805 22:46:38.604183 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:38.604410 24341 external_mini_cluster.cc:1428] Setting key 6f731d48b1f6b5e4cfb3c95a0b45512e
I20260805 22:46:38.607237 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:38.604979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:42529 (local address 127.0.0.1:46261)
0805 22:46:38.605332 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:38.605343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:38.605376 (+ 33us) server_negotiation.cc:407] Connection header received
0805 22:46:38.605858 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:38.605867 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:38.605901 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:38.606056 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:38.606787 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:38.606792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:38.606792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:38.606797 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:38.606842 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:38.606855 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:38.606933 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:38.606993 (+ 60us) server_negotiation.cc:299] Negotiation successful
0805 22:46:38.607036 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":77,"threads_started":1}
I20260805 22:46:38.607700 23825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f: Bootstrap starting.
I20260805 22:46:38.608307 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:38.608575 23825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:38.608928 23825 log.cc:824] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:38.609728 23825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f: No bootstrap required, opened a new log
I20260805 22:46:38.610996 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:38.610065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:57271 (local address 127.0.0.1:46261)
0805 22:46:38.610118 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:38.610120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:38.610128 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:38.610148 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:38.610149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:38.610154 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:38.610218 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:38.610785 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:38.610786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:38.610787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:38.610791 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:38.610805 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:38.610810 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:38.610883 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:38.610918 (+ 35us) server_negotiation.cc:299] Negotiation successful
0805 22:46:38.610932 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 22:46:38.612030 23825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859dd0ec1a704046add1af1fac22aa0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } }
I20260805 22:46:38.612164 23825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:38.612186 23825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 859dd0ec1a704046add1af1fac22aa0f, State: Initialized, Role: FOLLOWER
I20260805 22:46:38.612327 23825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859dd0ec1a704046add1af1fac22aa0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } }
I20260805 22:46:38.612417 23825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:38.612462 23825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:38.612514 23825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:38.613153 23825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859dd0ec1a704046add1af1fac22aa0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } }
I20260805 22:46:38.613248 23825 leader_election.cc:302] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 859dd0ec1a704046add1af1fac22aa0f; no voters:
I20260805 22:46:38.613435 23825 leader_election.cc:288] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:38.613487 23843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:38.613703 23825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:38.613723 23843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [term 1 LEADER]: Becoming Leader. State: Replica: 859dd0ec1a704046add1af1fac22aa0f, State: Running, Role: LEADER
I20260805 22:46:38.613919 23843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859dd0ec1a704046add1af1fac22aa0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } }
I20260805 22:46:38.614279 23845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "859dd0ec1a704046add1af1fac22aa0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859dd0ec1a704046add1af1fac22aa0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } } }
I20260805 22:46:38.614291 23846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 859dd0ec1a704046add1af1fac22aa0f. Latest consensus state: current_term: 1 leader_uuid: "859dd0ec1a704046add1af1fac22aa0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859dd0ec1a704046add1af1fac22aa0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46261 } } }
I20260805 22:46:38.614460 23846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:38.614434 23845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:38.614658 23856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:38.615201 23856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:38.616827 23856 catalog_manager.cc:1347] Generated new cluster ID: 4158c9a171b448849c4fad8292e80054
I20260805 22:46:38.616884 23856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:38.626353 23856 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:38.626534 23856 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:38.631377 23856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 859dd0ec1a704046add1af1fac22aa0f: Generated new TSK 0
I20260805 22:46:38.631587 23856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:38.634277 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 23730
2026-08-05T22:46:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1591 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:38 dist-test-slave-g170 krb5kdc[23874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:38 dist-test-slave-g170 krb5kdc[23874](info): set up 2 sockets
Aug 05 22:46:38 dist-test-slave-g170 krb5kdc[23874](info): commencing operation
krb5kdc: starting...
W20260805 22:46:39.886034 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:39 dist-test-slave-g170 krb5kdc[23874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:40.020877 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:39633
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:39633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:40.118499 23893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:40.118731 23893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:40.118760 23893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:40.119988 23893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:40.120030 23893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:40.120048 23893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:40.120062 23893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:40.120076 23893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:40.121803 23893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:39633
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:39633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:40.122040 23893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:40.122242 23893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:40.127818 23904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:40.127871 23901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:40.127974 23893 server_base.cc:1034] running on GCE node
W20260805 22:46:40.127823 23902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:40.128309 23893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:40.128664 23893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:40.129864 23893 hybrid_clock.cc:648] HybridClock initialized: now 1785970000129845 us; error 44 us; skew 500 ppm
Aug 05 22:46:40 dist-test-slave-g170 krb5kdc[23874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970000, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:40.132365 23893 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:40.132925 23893 webserver.cc:460] Webserver started at http://127.23.197.126:38185/ using document root <none> and password file <none>
I20260805 22:46:40.133168 23893 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:40.133241 23893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:40.133400 23893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:40.134509 23893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "73f77ed8fe2348179565dbb36e1c2334"
format_stamp: "Formatted at 2026-08-05 22:46:40 on dist-test-slave-g170"
server_key: "25363a0edea70efaf29b04cacbd04c03"
server_key_iv: "18d70edbc87782549a4f8f9cab683be6"
server_key_version: "encryptionkey@0"
I20260805 22:46:40.134904 23893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "73f77ed8fe2348179565dbb36e1c2334"
format_stamp: "Formatted at 2026-08-05 22:46:40 on dist-test-slave-g170"
server_key: "25363a0edea70efaf29b04cacbd04c03"
server_key_iv: "18d70edbc87782549a4f8f9cab683be6"
server_key_version: "encryptionkey@0"
I20260805 22:46:40.136317 23893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:40.137313 23915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:40.137504 23893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:46:40.137614 23893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "73f77ed8fe2348179565dbb36e1c2334"
format_stamp: "Formatted at 2026-08-05 22:46:40 on dist-test-slave-g170"
server_key: "25363a0edea70efaf29b04cacbd04c03"
server_key_iv: "18d70edbc87782549a4f8f9cab683be6"
server_key_version: "encryptionkey@0"
I20260805 22:46:40.137742 23893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:40.159664 23893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:40.161863 23893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:40.162055 23893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:40.166497 23893 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:39633
I20260805 22:46:40.166541 23988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:39633 every 8 connection(s)
I20260805 22:46:40.167145 23893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:40.167938 23989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:40.170496 23989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334: Bootstrap starting.
I20260805 22:46:40.171094 23989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:40.171375 23989 log.cc:824] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:40.172113 23989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334: No bootstrap required, opened a new log
I20260805 22:46:40.173933 23989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f77ed8fe2348179565dbb36e1c2334" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39633 } }
I20260805 22:46:40.174080 23989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:40.174105 23989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f77ed8fe2348179565dbb36e1c2334, State: Initialized, Role: FOLLOWER
I20260805 22:46:40.174252 23989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f77ed8fe2348179565dbb36e1c2334" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39633 } }
I20260805 22:46:40.174346 23989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:40.174372 23989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:40.174413 23989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:40.175387 23989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f77ed8fe2348179565dbb36e1c2334" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39633 } }
I20260805 22:46:40.175482 23989 leader_election.cc:302] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73f77ed8fe2348179565dbb36e1c2334; no voters:
I20260805 22:46:40.175623 23989 leader_election.cc:288] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:40.175702 23996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:40.175863 23996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [term 1 LEADER]: Becoming Leader. State: Replica: 73f77ed8fe2348179565dbb36e1c2334, State: Running, Role: LEADER
I20260805 22:46:40.175901 23989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:40.176019 23996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f77ed8fe2348179565dbb36e1c2334" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39633 } }
I20260805 22:46:40.176404 24000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73f77ed8fe2348179565dbb36e1c2334. Latest consensus state: current_term: 1 leader_uuid: "73f77ed8fe2348179565dbb36e1c2334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f77ed8fe2348179565dbb36e1c2334" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39633 } } }
I20260805 22:46:40.176505 24000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:40.176399 23999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73f77ed8fe2348179565dbb36e1c2334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f77ed8fe2348179565dbb36e1c2334" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 39633 } } }
I20260805 22:46:40.176636 23999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:40.176857 24007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:40.177277 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 23893
I20260805 22:46:40.177412 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:40.177615 24341 external_mini_cluster.cc:1428] Setting key 0f1c1024f48d24d0d8b12ee0e1fa6629
I20260805 22:46:40.178228 24007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:40.180334 24007 catalog_manager.cc:1347] Generated new cluster ID: eb39e90475d2430db4213b39d9ba8b50
I20260805 22:46:40.180390 24007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 05 22:46:40 dist-test-slave-g170 krb5kdc[23874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785969999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:40.189131 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:40.178172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35589 (local address 127.23.197.126:39633)
0805 22:46:40.178566 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:40.178578 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:40.178622 (+ 44us) server_negotiation.cc:407] Connection header received
0805 22:46:40.178936 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:40.178942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:40.179024 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:40.179177 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:40.179980 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:40.181272 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:40.182158 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:40.182622 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:40.182756 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:40.185427 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:40.185432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:40.185432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:40.185445 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:40.187335 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:40.187798 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:40.187801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:40.187801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:40.187905 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:40.188172 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:40.188174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:40.188174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:40.188435 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:40.188636 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:40.188785 (+ 149us) server_negotiation.cc:299] Negotiation successful
0805 22:46:40.188915 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":91,"threads_started":1}
I20260805 22:46:40.190173 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:40.192484 24007 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:40.193214 24007 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:40.198627 24007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73f77ed8fe2348179565dbb36e1c2334: Generated new TSK 0
I20260805 22:46:40.198884 24007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:40.199098 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:40.191687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43971 (local address 127.23.197.126:39633)
0805 22:46:40.191728 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:40.191730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:40.191960 (+ 230us) server_negotiation.cc:407] Connection header received
0805 22:46:40.192058 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:40.192059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:40.192070 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:40.192101 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:40.192971 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:40.193599 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:40.194377 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:40.194614 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:40.194699 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:40.195972 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:40.195973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:40.195974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:40.195982 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:40.197591 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:40.198003 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:40.198005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:40.198005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:40.198072 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:40.198365 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:40.198366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:40.198366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:40.198530 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:40.198640 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:40.198916 (+ 276us) server_negotiation.cc:299] Negotiation successful
0805 22:46:40.198990 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260805 22:46:40.201232 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 23893
W20260805 22:46:40.209030 23872 connection.cc:535] client connection to 127.23.197.126:39633 recv error: Network error: failed to read from TLS socket (remote: 127.23.197.126:39633): Connection reset by peer (error 104)
2026-08-05T22:46:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1562 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:40 dist-test-slave-g170 krb5kdc[24034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:40 dist-test-slave-g170 krb5kdc[24034](info): set up 2 sockets
Aug 05 22:46:40 dist-test-slave-g170 krb5kdc[24034](info): commencing operation
krb5kdc: starting...
W20260805 22:46:41.489226 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:41 dist-test-slave-g170 krb5kdc[24034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:41.621345 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:35765
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:35765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:41.713618 24053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:41.713845 24053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:41.713868 24053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:41.715008 24053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:41.715052 24053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:41.715066 24053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:41.715080 24053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:41.715091 24053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:41.716585 24053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:35765
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:35765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:41.716874 24053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:41.717128 24053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:41.722528 24061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:41.722638 24063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:41.722536 24060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:41.722945 24053 server_base.cc:1034] running on GCE node
I20260805 22:46:41.723241 24053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:41.723541 24053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:41.724720 24053 hybrid_clock.cc:648] HybridClock initialized: now 1785970001724704 us; error 29 us; skew 500 ppm
Aug 05 22:46:41 dist-test-slave-g170 krb5kdc[24034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970001, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:41.732271 24053 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:41.732728 24053 webserver.cc:460] Webserver started at http://127.23.197.126:42101/ using document root <none> and password file <none>
I20260805 22:46:41.733045 24053 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:41.733122 24053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:41.733326 24053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:41.734597 24053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "84084b5ceaa443bfa234bd7a177ca78f"
format_stamp: "Formatted at 2026-08-05 22:46:41 on dist-test-slave-g170"
server_key: "45e47605ca07efaed029754428e0771f"
server_key_iv: "a7551e7b86ed39782c7244e08bcd078e"
server_key_version: "encryptionkey@0"
I20260805 22:46:41.735008 24053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "84084b5ceaa443bfa234bd7a177ca78f"
format_stamp: "Formatted at 2026-08-05 22:46:41 on dist-test-slave-g170"
server_key: "45e47605ca07efaed029754428e0771f"
server_key_iv: "a7551e7b86ed39782c7244e08bcd078e"
server_key_version: "encryptionkey@0"
I20260805 22:46:41.736560 24053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260805 22:46:41.737483 24073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:41.737721 24053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260805 22:46:41.737852 24053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "84084b5ceaa443bfa234bd7a177ca78f"
format_stamp: "Formatted at 2026-08-05 22:46:41 on dist-test-slave-g170"
server_key: "45e47605ca07efaed029754428e0771f"
server_key_iv: "a7551e7b86ed39782c7244e08bcd078e"
server_key_version: "encryptionkey@0"
I20260805 22:46:41.737931 24053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:41.751893 24053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:41.754511 24053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:41.754678 24053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:41.759208 24053 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:35765
I20260805 22:46:41.759249 24137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:35765 every 8 connection(s)
I20260805 22:46:41.759634 24053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:41.760533 24138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:41.762923 24138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f: Bootstrap starting.
I20260805 22:46:41.763528 24138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:41.763833 24138 log.cc:824] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:41.764552 24138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f: No bootstrap required, opened a new log
I20260805 22:46:41.766213 24138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84084b5ceaa443bfa234bd7a177ca78f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35765 } }
I20260805 22:46:41.766352 24138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:41.766374 24138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84084b5ceaa443bfa234bd7a177ca78f, State: Initialized, Role: FOLLOWER
I20260805 22:46:41.766557 24138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84084b5ceaa443bfa234bd7a177ca78f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35765 } }
I20260805 22:46:41.766671 24138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:41.766726 24138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:41.766776 24138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:41.767673 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24053
I20260805 22:46:41.767684 24138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84084b5ceaa443bfa234bd7a177ca78f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35765 } }
I20260805 22:46:41.767818 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:41.767798 24138 leader_election.cc:302] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84084b5ceaa443bfa234bd7a177ca78f; no voters:
I20260805 22:46:41.767985 24138 leader_election.cc:288] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:41.768030 24341 external_mini_cluster.cc:1428] Setting key 6fce5c2fe02dc584fa035f6e02ca5d35
I20260805 22:46:41.768011 24143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:41.768195 24143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [term 1 LEADER]: Becoming Leader. State: Replica: 84084b5ceaa443bfa234bd7a177ca78f, State: Running, Role: LEADER
I20260805 22:46:41.768271 24143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84084b5ceaa443bfa234bd7a177ca78f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35765 } }
I20260805 22:46:41.768289 24138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:41.769357 24146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84084b5ceaa443bfa234bd7a177ca78f. Latest consensus state: current_term: 1 leader_uuid: "84084b5ceaa443bfa234bd7a177ca78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84084b5ceaa443bfa234bd7a177ca78f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35765 } } }
I20260805 22:46:41.769390 24144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84084b5ceaa443bfa234bd7a177ca78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84084b5ceaa443bfa234bd7a177ca78f" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 35765 } } }
I20260805 22:46:41.769479 24146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:41.769491 24144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:41.769757 24159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:41.770385 24159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:41.772048 24159 catalog_manager.cc:1347] Generated new cluster ID: b072c70899594efc9a8a4ebfac8c3bf9
I20260805 22:46:41.772101 24159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:41.775173 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:41.768867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33619 (local address 127.23.197.126:35765)
0805 22:46:41.769474 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:41.769484 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:41.769520 (+ 36us) server_negotiation.cc:407] Connection header received
0805 22:46:41.769645 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:41.769652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:41.769726 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:41.769945 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:41.770209 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:41.771737 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:41.772909 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:41.773542 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:41.773680 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:41.774428 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:41.774432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:41.774433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:41.774674 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:41.774713 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:41.774729 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:41.774819 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:41.774912 (+ 93us) server_negotiation.cc:299] Negotiation successful
0805 22:46:41.774941 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":122,"threads_started":1}
I20260805 22:46:41.776116 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:41.779512 24159 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:41.780004 24159 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:41.781644 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:41.777733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56015 (local address 127.23.197.126:35765)
0805 22:46:41.777775 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:41.777777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:41.777912 (+ 135us) server_negotiation.cc:407] Connection header received
0805 22:46:41.778017 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:41.778018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:41.778030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:41.778060 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:41.778280 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:41.779080 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:41.780134 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:41.780435 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:41.780476 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:41.781355 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:41.781357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:41.781358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:41.781362 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:41.781376 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:41.781380 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:41.781413 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:41.781542 (+ 129us) server_negotiation.cc:299] Negotiation successful
0805 22:46:41.781556 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260805 22:46:41.782719 24159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84084b5ceaa443bfa234bd7a177ca78f: Generated new TSK 0
I20260805 22:46:41.782887 24159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:41.789486 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24053
W20260805 22:46:41.810122 24032 connection.cc:535] client connection to 127.23.197.126:35765 recv error: Network error: failed to read from TLS socket (remote: 127.23.197.126:35765): Connection reset by peer (error 104)
2026-08-05T22:46:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1600 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:41 dist-test-slave-g170 krb5kdc[24188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:41 dist-test-slave-g170 krb5kdc[24188](info): set up 2 sockets
Aug 05 22:46:41 dist-test-slave-g170 krb5kdc[24188](info): commencing operation
krb5kdc: starting...
W20260805 22:46:43.138489 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:43 dist-test-slave-g170 krb5kdc[24188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:43.268327 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36535
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:45789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:36535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:43.352612 24205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:43.352880 24205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:43.352919 24205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:43.354069 24205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:43.354139 24205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:43.354164 24205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:43.354203 24205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:43.354233 24205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:43.355690 24205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:36535
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:36535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:43.355963 24205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:43.356199 24205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:43.361078 24216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:43.361104 24213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:43.361104 24212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:43.361228 24205 server_base.cc:1034] running on GCE node
I20260805 22:46:43.361738 24205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:43.362033 24205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:43.363194 24205 hybrid_clock.cc:648] HybridClock initialized: now 1785970003363179 us; error 30 us; skew 500 ppm
Aug 05 22:46:43 dist-test-slave-g170 krb5kdc[24188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:43.365679 24205 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:43.366111 24205 webserver.cc:460] Webserver started at http://127.23.197.126:40399/ using document root <none> and password file <none>
I20260805 22:46:43.366307 24205 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:43.366348 24205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:43.366438 24205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:43.367416 24205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a"
format_stamp: "Formatted at 2026-08-05 22:46:43 on dist-test-slave-g170"
server_key: "cb794fa936ab1612bd444a658e78c6e3"
server_key_iv: "a96ad23eeafe9919e20e5b5bab66b4a6"
server_key_version: "encryptionkey@0"
I20260805 22:46:43.367801 24205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a"
format_stamp: "Formatted at 2026-08-05 22:46:43 on dist-test-slave-g170"
server_key: "cb794fa936ab1612bd444a658e78c6e3"
server_key_iv: "a96ad23eeafe9919e20e5b5bab66b4a6"
server_key_version: "encryptionkey@0"
I20260805 22:46:43.369195 24205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260805 22:46:43.369911 24226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:43.370117 24205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:43.370229 24205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a"
format_stamp: "Formatted at 2026-08-05 22:46:43 on dist-test-slave-g170"
server_key: "cb794fa936ab1612bd444a658e78c6e3"
server_key_iv: "a96ad23eeafe9919e20e5b5bab66b4a6"
server_key_version: "encryptionkey@0"
I20260805 22:46:43.370285 24205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:43.373615 24205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:43.375893 24205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:43.376003 24205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:43.380259 24205 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:36535
I20260805 22:46:43.380291 24297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:36535 every 8 connection(s)
I20260805 22:46:43.380635 24205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:43.381724 24300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:43.383705 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24205
I20260805 22:46:43.383859 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:43.384058 24341 external_mini_cluster.cc:1428] Setting key e15365831c813c38976e604fa452ecc9
I20260805 22:46:43.384334 24300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a: Bootstrap starting.
I20260805 22:46:43.385135 24300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:43.385496 24300 log.cc:824] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:43.386365 24300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a: No bootstrap required, opened a new log
I20260805 22:46:43.387113 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:43.384646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32999 (local address 127.23.197.126:36535)
0805 22:46:43.384884 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:43.384894 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:43.384930 (+ 36us) server_negotiation.cc:407] Connection header received
0805 22:46:43.385587 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:43.385596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:43.385631 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:43.385804 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:43.386601 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:43.386606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:43.386606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:43.386612 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:43.386672 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:43.386683 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:43.386792 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:43.386852 (+ 60us) server_negotiation.cc:299] Negotiation successful
0805 22:46:43.386894 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":125,"threads_started":1}
I20260805 22:46:43.388260 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:43.389084 24300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36535 } }
I20260805 22:46:43.389267 24300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:43.389323 24300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3b4d77ebd842b49cf7f1ee8bb15e4a, State: Initialized, Role: FOLLOWER
I20260805 22:46:43.389492 24300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36535 } }
I20260805 22:46:43.389586 24300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:43.389643 24300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:43.389704 24300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:43.390611 24300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36535 } }
I20260805 22:46:43.390740 24300 leader_election.cc:302] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a3b4d77ebd842b49cf7f1ee8bb15e4a; no voters:
I20260805 22:46:43.390949 24300 leader_election.cc:288] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:43.391005 24314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:43.391152 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:43.389912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37555 (local address 127.23.197.126:36535)
0805 22:46:43.389958 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:43.389960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:43.390082 (+ 122us) server_negotiation.cc:407] Connection header received
0805 22:46:43.390160 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:43.390160 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:43.390164 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:43.390212 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:43.390846 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:43.390848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:43.390848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:43.390855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:43.390871 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:43.390876 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:43.390981 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:43.391067 (+ 86us) server_negotiation.cc:299] Negotiation successful
0805 22:46:43.391091 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:46:43.391263 24314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [term 1 LEADER]: Becoming Leader. State: Replica: 0a3b4d77ebd842b49cf7f1ee8bb15e4a, State: Running, Role: LEADER
I20260805 22:46:43.391299 24300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:43.391587 24314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36535 } }
I20260805 22:46:43.391970 24314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a3b4d77ebd842b49cf7f1ee8bb15e4a. Latest consensus state: current_term: 1 leader_uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36535 } } }
I20260805 22:46:43.392009 24315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3b4d77ebd842b49cf7f1ee8bb15e4a" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 36535 } } }
I20260805 22:46:43.392133 24314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:43.392151 24315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:43.392609 24331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:43.393263 24331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:43.394905 24331 catalog_manager.cc:1347] Generated new cluster ID: 1d765dd2787f4f79aded07722ee68375
I20260805 22:46:43.394972 24331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:43.415340 24331 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:43.415556 24331 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:43.424364 24331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a3b4d77ebd842b49cf7f1ee8bb15e4a: Generated new TSK 0
I20260805 22:46:43.424603 24331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:43.427675 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24205
2026-08-05T22:46:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1625 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:43 dist-test-slave-g170 krb5kdc[24350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:43 dist-test-slave-g170 krb5kdc[24350](info): set up 2 sockets
Aug 05 22:46:43 dist-test-slave-g170 krb5kdc[24350](info): commencing operation
krb5kdc: starting...
W20260805 22:46:44.669168 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:44 dist-test-slave-g170 krb5kdc[24350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:44.800328 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33829
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:45179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:33829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:44.885218 24374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:44.885426 24374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:44.885450 24374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:44.886623 24374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:44.886670 24374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:44.886684 24374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:44.886698 24374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:44.886736 24374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:44.888249 24374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:45179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:33829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:33829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:44.888510 24374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:44.888746 24374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:44.893738 24385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:44.893785 24380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:44.893735 24382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:44.894004 24374 server_base.cc:1034] running on GCE node
I20260805 22:46:44.894347 24374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:44.894690 24374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:44.895889 24374 hybrid_clock.cc:648] HybridClock initialized: now 1785970004895865 us; error 42 us; skew 500 ppm
Aug 05 22:46:44 dist-test-slave-g170 krb5kdc[24350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:44.898780 24374 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:44.899250 24374 webserver.cc:460] Webserver started at http://127.23.197.126:32815/ using document root <none> and password file <none>
I20260805 22:46:44.899497 24374 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:44.899569 24374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:44.899714 24374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:44.900902 24374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "18f453ad58944cc2adf3524400a712f6"
format_stamp: "Formatted at 2026-08-05 22:46:44 on dist-test-slave-g170"
server_key: "037146195f52caf6de1b28bbb6e00862"
server_key_iv: "3f860785ca70c214a685da564fcca941"
server_key_version: "encryptionkey@0"
I20260805 22:46:44.901278 24374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "18f453ad58944cc2adf3524400a712f6"
format_stamp: "Formatted at 2026-08-05 22:46:44 on dist-test-slave-g170"
server_key: "037146195f52caf6de1b28bbb6e00862"
server_key_iv: "3f860785ca70c214a685da564fcca941"
server_key_version: "encryptionkey@0"
I20260805 22:46:44.902666 24374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 22:46:44.903426 24397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:44.903601 24374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260805 22:46:44.903712 24374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "18f453ad58944cc2adf3524400a712f6"
format_stamp: "Formatted at 2026-08-05 22:46:44 on dist-test-slave-g170"
server_key: "037146195f52caf6de1b28bbb6e00862"
server_key_iv: "3f860785ca70c214a685da564fcca941"
server_key_version: "encryptionkey@0"
I20260805 22:46:44.903786 24374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:44.915019 24374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:44.917181 24374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:44.917354 24374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:44.921625 24374 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:33829
I20260805 22:46:44.921696 24460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:33829 every 8 connection(s)
I20260805 22:46:44.922077 24374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:44.923038 24462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:44.925505 24462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6: Bootstrap starting.
I20260805 22:46:44.926188 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24374
I20260805 22:46:44.926231 24462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:44.926326 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:44.926569 24341 external_mini_cluster.cc:1428] Setting key 295b6c337578e0dcf43102919cca2248
I20260805 22:46:44.926600 24462 log.cc:824] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:44.927474 24462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6: No bootstrap required, opened a new log
I20260805 22:46:44.929695 24462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f453ad58944cc2adf3524400a712f6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33829 } }
I20260805 22:46:44.929867 24462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:44.929919 24462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f453ad58944cc2adf3524400a712f6, State: Initialized, Role: FOLLOWER
I20260805 22:46:44.930070 24462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f453ad58944cc2adf3524400a712f6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33829 } }
I20260805 22:46:44.930169 24462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:44.930214 24462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:44.930262 24462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:44.930855 24462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f453ad58944cc2adf3524400a712f6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33829 } }
I20260805 22:46:44.930976 24462 leader_election.cc:302] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18f453ad58944cc2adf3524400a712f6; no voters:
I20260805 22:46:44.931205 24471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:44.931180 24462 leader_election.cc:288] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:44.931407 24471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [term 1 LEADER]: Becoming Leader. State: Replica: 18f453ad58944cc2adf3524400a712f6, State: Running, Role: LEADER
I20260805 22:46:44.931535 24471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f453ad58944cc2adf3524400a712f6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33829 } }
I20260805 22:46:44.931710 24462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:44.931915 24475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18f453ad58944cc2adf3524400a712f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f453ad58944cc2adf3524400a712f6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33829 } } }
I20260805 22:46:44.932047 24475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:44.931950 24474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18f453ad58944cc2adf3524400a712f6. Latest consensus state: current_term: 1 leader_uuid: "18f453ad58944cc2adf3524400a712f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f453ad58944cc2adf3524400a712f6" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 33829 } } }
I20260805 22:46:44.932289 24474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:44.932291 24478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:44.932958 24478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 05 22:46:44 dist-test-slave-g170 krb5kdc[24350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:44.938145 24478 catalog_manager.cc:1347] Generated new cluster ID: 59b5d9423878474e8cc2e13fe311e42e
I20260805 22:46:44.938205 24478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:44.938527 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:44.927109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52939 (local address 127.23.197.126:33829)
0805 22:46:44.927327 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:44.927338 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:44.927375 (+ 37us) server_negotiation.cc:407] Connection header received
0805 22:46:44.927861 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:44.927869 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:44.927920 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:44.928070 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:44.928743 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:44.929689 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:44.930655 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:44.931168 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:44.931292 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:44.934533 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:44.934538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:44.934538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:44.934552 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:44.936535 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:44.937021 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:44.937025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:44.937025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:44.937126 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:44.937352 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:44.937353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:44.937354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:44.937692 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:44.937869 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:44.938152 (+ 283us) server_negotiation.cc:299] Negotiation successful
0805 22:46:44.938299 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260805 22:46:44.943905 24478 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:44.944404 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:44.944653 24478 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:44.953517 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:44.946028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36325 (local address 127.23.197.126:33829)
0805 22:46:44.946071 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:44.946073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:44.946163 (+ 90us) server_negotiation.cc:407] Connection header received
0805 22:46:44.946222 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:44.946222 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:44.946232 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:44.946282 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:44.946947 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:44.947717 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:44.948816 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:44.949117 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:44.949178 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:44.950620 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:44.950621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:44.950621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:44.950628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:44.952197 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:44.952592 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:44.952592 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:44.952593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:44.952654 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:44.952910 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:44.952910 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:44.952911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:44.953027 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:44.953115 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:44.953256 (+ 141us) server_negotiation.cc:299] Negotiation successful
0805 22:46:44.953336 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":23}
I20260805 22:46:44.955883 24478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18f453ad58944cc2adf3524400a712f6: Generated new TSK 0
I20260805 22:46:44.956064 24478 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:44.957974 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24374
2026-08-05T22:46:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:45 dist-test-slave-g170 krb5kdc[24511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:45 dist-test-slave-g170 krb5kdc[24511](info): set up 2 sockets
Aug 05 22:46:45 dist-test-slave-g170 krb5kdc[24511](info): commencing operation
krb5kdc: starting...
W20260805 22:46:46.325489 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:46 dist-test-slave-g170 krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:46.455363 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:37631
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:43333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:37631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:46.544045 24527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:46.544306 24527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:46.544333 24527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:46.545552 24527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:46.545611 24527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:46.545627 24527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:46.545642 24527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:46.545656 24527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:46.547151 24527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:43333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:37631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:37631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:46.547363 24527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:46.547557 24527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:46.552925 24533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:46.552942 24536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:46.552920 24534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:46.553390 24527 server_base.cc:1034] running on GCE node
I20260805 22:46:46.553680 24527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:46.554023 24527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:46.555238 24527 hybrid_clock.cc:648] HybridClock initialized: now 1785970006555221 us; error 28 us; skew 500 ppm
Aug 05 22:46:46 dist-test-slave-g170 krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970006, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:46.557932 24527 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:46.558355 24527 webserver.cc:460] Webserver started at http://127.23.197.126:34919/ using document root <none> and password file <none>
I20260805 22:46:46.558561 24527 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:46.558604 24527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:46.558696 24527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:46.559719 24527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35"
format_stamp: "Formatted at 2026-08-05 22:46:46 on dist-test-slave-g170"
server_key: "2beb13076381e71bed3843b19d9e4fca"
server_key_iv: "b870d548f041482f2ffe1dd234c8dcd1"
server_key_version: "encryptionkey@0"
I20260805 22:46:46.560029 24527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35"
format_stamp: "Formatted at 2026-08-05 22:46:46 on dist-test-slave-g170"
server_key: "2beb13076381e71bed3843b19d9e4fca"
server_key_iv: "b870d548f041482f2ffe1dd234c8dcd1"
server_key_version: "encryptionkey@0"
I20260805 22:46:46.561496 24527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260805 22:46:46.562290 24550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:46.562494 24527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:46.562588 24527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35"
format_stamp: "Formatted at 2026-08-05 22:46:46 on dist-test-slave-g170"
server_key: "2beb13076381e71bed3843b19d9e4fca"
server_key_iv: "b870d548f041482f2ffe1dd234c8dcd1"
server_key_version: "encryptionkey@0"
I20260805 22:46:46.562644 24527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:46.585556 24527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:46.588100 24527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:46.588227 24527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:46.592542 24527 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:37631
I20260805 22:46:46.592660 24625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:37631 every 8 connection(s)
I20260805 22:46:46.593019 24527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:46.594038 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:46.596587 24627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35: Bootstrap starting.
I20260805 22:46:46.597283 24627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:46.597636 24627 log.cc:824] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:46.598500 24627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35: No bootstrap required, opened a new log
I20260805 22:46:46.600293 24627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37631 } }
I20260805 22:46:46.600441 24627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:46.600474 24627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ccd2d3384114ed6b3a19ba2a7c1bd35, State: Initialized, Role: FOLLOWER
I20260805 22:46:46.600689 24627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37631 } }
I20260805 22:46:46.600829 24627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:46.600879 24627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:46.600937 24627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:46.601617 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24527
I20260805 22:46:46.601792 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:46.601996 24341 external_mini_cluster.cc:1428] Setting key 01c1392d49abcd31c712699bb7b465e0
I20260805 22:46:46.602170 24627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37631 } }
I20260805 22:46:46.602308 24627 leader_election.cc:302] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ccd2d3384114ed6b3a19ba2a7c1bd35; no voters:
I20260805 22:46:46.602504 24627 leader_election.cc:288] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:46.602521 24631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:46.602873 24631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [term 1 LEADER]: Becoming Leader. State: Replica: 2ccd2d3384114ed6b3a19ba2a7c1bd35, State: Running, Role: LEADER
I20260805 22:46:46.602986 24627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:46.603046 24631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37631 } }
I20260805 22:46:46.603447 24631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ccd2d3384114ed6b3a19ba2a7c1bd35. Latest consensus state: current_term: 1 leader_uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37631 } } }
I20260805 22:46:46.603511 24636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccd2d3384114ed6b3a19ba2a7c1bd35" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37631 } } }
I20260805 22:46:46.603605 24636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:46.603829 24631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:46.604260 24643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:46.604880 24643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:46.606598 24643 catalog_manager.cc:1347] Generated new cluster ID: ba3cca11ab7e49b387fac66a100573e3
I20260805 22:46:46.606660 24643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:46.608804 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:46.602549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44671 (local address 127.23.197.126:37631)
0805 22:46:46.602739 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:46.602751 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:46.603005 (+ 254us) server_negotiation.cc:407] Connection header received
0805 22:46:46.603129 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:46.603163 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:46.603229 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:46.603390 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:46.603801 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:46.605458 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:46.606512 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:46.607101 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:46.607225 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:46.608022 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:46.608025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:46.608026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:46.608241 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:46.608279 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:46.608293 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:46.608354 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:46.608473 (+ 119us) server_negotiation.cc:299] Negotiation successful
0805 22:46:46.608533 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260805 22:46:46.609763 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:46.615242 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:46.611453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46491 (local address 127.23.197.126:37631)
0805 22:46:46.611495 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:46.611497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:46.611645 (+ 148us) server_negotiation.cc:407] Connection header received
0805 22:46:46.611708 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:46.611709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:46.611731 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:46.611782 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:46.611920 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:46.612636 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:46.613897 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:46.614194 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:46.614254 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:46.614983 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:46.614984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:46.614984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:46.614990 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:46.615006 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:46.615010 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:46.615063 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:46.615146 (+ 83us) server_negotiation.cc:299] Negotiation successful
0805 22:46:46.615169 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260805 22:46:46.637370 24643 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:46.638008 24643 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:46.645329 24643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ccd2d3384114ed6b3a19ba2a7c1bd35: Generated new TSK 0
I20260805 22:46:46.645562 24643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:46.656865 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24527
2026-08-05T22:46:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1697 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:46 dist-test-slave-g170 krb5kdc[24674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:46 dist-test-slave-g170 krb5kdc[24674](info): set up 2 sockets
Aug 05 22:46:46 dist-test-slave-g170 krb5kdc[24674](info): commencing operation
krb5kdc: starting...
W20260805 22:46:47.978662 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:48 dist-test-slave-g170 krb5kdc[24674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:48.108745 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:37605
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:38085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:48.192752 24693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:48.193102 24693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:48.193145 24693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:48.194300 24693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:48.194376 24693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:48.194401 24693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:48.194420 24693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:48.194453 24693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:48.195988 24693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:38085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:37605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:37605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:48.196286 24693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:48.196527 24693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:48.201686 24700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:48.201809 24699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:48.201731 24702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:48.201939 24693 server_base.cc:1034] running on GCE node
I20260805 22:46:48.202268 24693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:48.202575 24693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:48.203756 24693 hybrid_clock.cc:648] HybridClock initialized: now 1785970008203727 us; error 45 us; skew 500 ppm
Aug 05 22:46:48 dist-test-slave-g170 krb5kdc[24674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970008, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:48.210472 24693 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:48.210924 24693 webserver.cc:460] Webserver started at http://127.23.197.126:41577/ using document root <none> and password file <none>
I20260805 22:46:48.211181 24693 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:48.211253 24693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:48.211397 24693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:48.212467 24693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "b27a7080b6564e729868f1fb5e84291e"
format_stamp: "Formatted at 2026-08-05 22:46:48 on dist-test-slave-g170"
server_key: "ef7e667a9de7f32cc127b8f337e96b5e"
server_key_iv: "a075febee008c8a203ec3100a88a6775"
server_key_version: "encryptionkey@0"
I20260805 22:46:48.212873 24693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "b27a7080b6564e729868f1fb5e84291e"
format_stamp: "Formatted at 2026-08-05 22:46:48 on dist-test-slave-g170"
server_key: "ef7e667a9de7f32cc127b8f337e96b5e"
server_key_iv: "a075febee008c8a203ec3100a88a6775"
server_key_version: "encryptionkey@0"
I20260805 22:46:48.214290 24693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:48.215047 24713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:48.215245 24693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260805 22:46:48.215344 24693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "b27a7080b6564e729868f1fb5e84291e"
format_stamp: "Formatted at 2026-08-05 22:46:48 on dist-test-slave-g170"
server_key: "ef7e667a9de7f32cc127b8f337e96b5e"
server_key_iv: "a075febee008c8a203ec3100a88a6775"
server_key_version: "encryptionkey@0"
I20260805 22:46:48.215421 24693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:48.218758 24693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:48.220674 24693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:48.220867 24693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:48.225111 24693 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:37605
I20260805 22:46:48.225160 24795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:37605 every 8 connection(s)
I20260805 22:46:48.225512 24693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:48.226478 24796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:48.228919 24796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e: Bootstrap starting.
I20260805 22:46:48.229528 24796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:48.229847 24796 log.cc:824] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:48.230580 24796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e: No bootstrap required, opened a new log
I20260805 22:46:48.232213 24796 raft_consensus.cc:348] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27a7080b6564e729868f1fb5e84291e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37605 } }
I20260805 22:46:48.232357 24796 raft_consensus.cc:374] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 22:46:48.232379 24796 raft_consensus.cc:729] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27a7080b6564e729868f1fb5e84291e, State: Initialized, Role: FOLLOWER
I20260805 22:46:48.232558 24796 consensus_queue.cc:260] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27a7080b6564e729868f1fb5e84291e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37605 } }
I20260805 22:46:48.232623 24796 raft_consensus.cc:388] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 22:46:48.232645 24796 raft_consensus.cc:482] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 22:46:48.232672 24796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:48.233603 24796 raft_consensus.cc:504] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27a7080b6564e729868f1fb5e84291e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37605 } }
I20260805 22:46:48.233732 24796 leader_election.cc:302] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b27a7080b6564e729868f1fb5e84291e; no voters:
I20260805 22:46:48.233925 24796 leader_election.cc:288] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 22:46:48.233978 24800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:48.234182 24800 raft_consensus.cc:686] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [term 1 LEADER]: Becoming Leader. State: Replica: b27a7080b6564e729868f1fb5e84291e, State: Running, Role: LEADER
I20260805 22:46:48.234226 24796 sys_catalog.cc:564] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:48.234315 24800 consensus_queue.cc:237] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27a7080b6564e729868f1fb5e84291e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37605 } }
I20260805 22:46:48.234535 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24693
I20260805 22:46:48.234671 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:48.234905 24341 external_mini_cluster.cc:1428] Setting key c5544c50b7cdd906eb0d92d91dc34174
I20260805 22:46:48.234843 24802 sys_catalog.cc:455] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b27a7080b6564e729868f1fb5e84291e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27a7080b6564e729868f1fb5e84291e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37605 } } }
I20260805 22:46:48.235006 24802 sys_catalog.cc:458] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:48.234846 24804 sys_catalog.cc:455] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b27a7080b6564e729868f1fb5e84291e. Latest consensus state: current_term: 1 leader_uuid: "b27a7080b6564e729868f1fb5e84291e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27a7080b6564e729868f1fb5e84291e" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 37605 } } }
I20260805 22:46:48.235602 24804 sys_catalog.cc:458] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e [sys.catalog]: This master's current role is: LEADER
W20260805 22:46:48.236480 24832 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:48.236614 24832 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:48.236747 24830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:48.236979 24830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:48.237809 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:48.235463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37359 (local address 127.23.197.126:37605)
0805 22:46:48.235921 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:48.236032 (+ 111us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:48.236083 (+ 51us) server_negotiation.cc:407] Connection header received
0805 22:46:48.236206 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:48.236214 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:48.236251 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:48.236415 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:48.237310 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:48.237315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:48.237315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:48.237321 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:48.237370 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:48.237387 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:48.237478 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:48.237559 (+ 81us) server_negotiation.cc:299] Negotiation successful
0805 22:46:48.237607 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":85,"threads_started":1}
I20260805 22:46:48.238938 24830 catalog_manager.cc:1347] Generated new cluster ID: cee0872dc2ba4d73b3f1d3e6564d2cbc
I20260805 22:46:48.239003 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:48.239032 24830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:48.245792 24830 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:48.245973 24830 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:48.246224 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:48.241079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34023 (local address 127.23.197.126:37605)
0805 22:46:48.241118 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:48.241120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:48.244970 (+ 3850us) server_negotiation.cc:407] Connection header received
0805 22:46:48.245107 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:48.245108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:48.245115 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:48.245185 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:48.245927 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:48.245929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:48.245929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 22:46:48.245934 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:48.245951 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 22:46:48.245957 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:48.245993 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:48.246140 (+ 147us) server_negotiation.cc:299] Negotiation successful
0805 22:46:48.246159 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260805 22:46:48.249358 24830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b27a7080b6564e729868f1fb5e84291e: Generated new TSK 0
I20260805 22:46:48.249545 24830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:48.257464 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24693
2026-08-05T22:46:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1608 ms)
[----------] 18 tests from AuthTokenIssuingTest (28732 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:48 dist-test-slave-g170 krb5kdc[24846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:48 dist-test-slave-g170 krb5kdc[24846](info): set up 2 sockets
Aug 05 22:46:48 dist-test-slave-g170 krb5kdc[24846](info): commencing operation
krb5kdc: starting...
W20260805 22:46:49.543469 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:49 dist-test-slave-g170 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:49.680234 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41855
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:41855,127.23.197.125:46625,127.23.197.124:38225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:49.767618 24867 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 22:46:49.767911 24867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:49.767943 24867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:49.767990 24867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:49.769261 24867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:49.769331 24867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:49.769354 24867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:49.769376 24867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:49.769405 24867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:49.770967 24867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:41855,127.23.197.125:46625,127.23.197.124:38225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:41855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:49.771278 24867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:49.771526 24867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:49.776742 24879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:49.776958 24883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:49.776714 24881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:49.776875 24867 server_base.cc:1034] running on GCE node
I20260805 22:46:49.777249 24867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:49.777665 24867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:49.778855 24867 hybrid_clock.cc:648] HybridClock initialized: now 1785970009778823 us; error 49 us; skew 500 ppm
Aug 05 22:46:49 dist-test-slave-g170 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:49.781497 24867 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:49.782001 24867 webserver.cc:460] Webserver started at http://127.23.197.126:42647/ using document root <none> and password file <none>
I20260805 22:46:49.782260 24867 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:49.782311 24867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:49.782451 24867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:49.783951 24867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "56cb4a79fe124aefb42e58703ac35519"
format_stamp: "Formatted at 2026-08-05 22:46:49 on dist-test-slave-g170"
server_key: "83a33fa7dbb3ef755dbcb31d99503db5"
server_key_iv: "36ba61a0b279ccd789e9252e42b4e587"
server_key_version: "encryptionkey@0"
I20260805 22:46:49.784411 24867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "56cb4a79fe124aefb42e58703ac35519"
format_stamp: "Formatted at 2026-08-05 22:46:49 on dist-test-slave-g170"
server_key: "83a33fa7dbb3ef755dbcb31d99503db5"
server_key_iv: "36ba61a0b279ccd789e9252e42b4e587"
server_key_version: "encryptionkey@0"
I20260805 22:46:49.786123 24867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260805 22:46:49.787070 24894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:49.787389 24867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:49.787499 24867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "56cb4a79fe124aefb42e58703ac35519"
format_stamp: "Formatted at 2026-08-05 22:46:49 on dist-test-slave-g170"
server_key: "83a33fa7dbb3ef755dbcb31d99503db5"
server_key_iv: "36ba61a0b279ccd789e9252e42b4e587"
server_key_version: "encryptionkey@0"
I20260805 22:46:49.787563 24867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:49.798308 24867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:49.800587 24867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:49.800712 24867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:49.804734 24867 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:41855
I20260805 22:46:49.804834 24962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:41855 every 8 connection(s)
I20260805 22:46:49.805218 24867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:49.806358 24966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:49.806867 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24867
I20260805 22:46:49.806964 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:49.807160 24341 external_mini_cluster.cc:1428] Setting key a989158df199c55f77969937b37a179f
I20260805 22:46:49.808375 24966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 22:46:49 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.197.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.125@KRBTEST.COM" created.
I20260805 22:46:49.819677 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.809317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.23.197.126:41855)
0805 22:46:49.809557 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:49.809563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:49.809664 (+ 101us) server_negotiation.cc:407] Connection header received
0805 22:46:49.810257 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:49.810260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:49.810282 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:49.810385 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:49.811067 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.811921 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.812886 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.813219 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.813356 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:49.815812 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:49.815821 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:49.815822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:49.815830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:49.817645 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:49.818221 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:49.818224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:49.818224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:49.818310 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:49.818568 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:49.818569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:49.818570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:49.818902 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:49.819007 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:49.819226 (+ 219us) server_negotiation.cc:299] Negotiation successful
0805 22:46:49.819408 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":206,"thread_start_us":110,"threads_started":1}
I20260805 22:46:49.819679 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.809289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41855 (local address 127.0.0.1:45680)
0805 22:46:49.809501 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.809551 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:49.810084 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:49.810443 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:49.810446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:49.810767 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:49.810934 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:49.810939 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.812050 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.812051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:49.812746 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:49.812748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.813325 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.813327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:49.813394 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:49.814072 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:49.814085 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:49.815689 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:49.817800 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:49.817809 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:49.817811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:49.818074 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:49.818400 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:49.818401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:49.818402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:49.818502 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:49.818983 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:49.818989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:49.819064 (+ 75us) client_negotiation.cc:769] Sending connection context
0805 22:46:49.819190 (+ 126us) client_negotiation.cc:240] Negotiation successful
0805 22:46:49.819457 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":116,"thread_start_us":112,"threads_started":1}
I20260805 22:46:49.820736 24966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:49.821223 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.821082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:46625 (local address 127.0.0.1:49012)
0805 22:46:49.821139 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.821197 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260805 22:46:49.821385 24899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
I20260805 22:46:49.821882 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.821756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:46625 (local address 127.0.0.1:49024)
0805 22:46:49.821827 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.821860 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260805 22:46:49.822049 24966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.125:46625: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab.
Entry for principal kudu/127.23.197.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab.
I20260805 22:46:49.836560 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.125:46625
--webserver_interface=127.23.197.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab
--principal=kudu/127.23.197.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:41855,127.23.197.125:46625,127.23.197.124:38225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:46:49.871361 24966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } attempt: 1
I20260805 22:46:49.871969 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.871815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:46625 (local address 127.0.0.1:49036)
0805 22:46:49.871891 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.871935 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260805 22:46:49.872352 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.872245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:46625 (local address 127.0.0.1:49044)
0805 22:46:49.872294 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.872320 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260805 22:46:49.872447 24966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.125:46625: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
W20260805 22:46:49.924043 24977 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 22:46:49.924342 24977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:49.924387 24977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:49.924431 24977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:49.925696 24977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:49.925767 24977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:49.925793 24977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:49.925846 24977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:49.925876 24977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:49.927399 24977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:41855,127.23.197.125:46625,127.23.197.124:38225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab
--principal=kudu/127.23.197.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.125:46625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.197.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:49.927678 24977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:49.927924 24977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:49.933068 24982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:49.933131 24983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:49.933131 24986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:49.933151 24977 server_base.cc:1034] running on GCE node
I20260805 22:46:49.933562 24977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:49.933868 24977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:49.935076 24977 hybrid_clock.cc:648] HybridClock initialized: now 1785970009935060 us; error 31 us; skew 500 ppm
Aug 05 22:46:49 dist-test-slave-g170 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:49.937775 24977 init.cc:377] Logged in from keytab as kudu/127.23.197.125@KRBTEST.COM (short username kudu)
I20260805 22:46:49.938236 24977 webserver.cc:460] Webserver started at http://127.23.197.125:44921/ using document root <none> and password file <none>
I20260805 22:46:49.938442 24977 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:49.938485 24977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:49.938581 24977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:49.939672 24977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/data/instance:
uuid: "5b3938a9110044f1b9e5d2e13240a0eb"
format_stamp: "Formatted at 2026-08-05 22:46:49 on dist-test-slave-g170"
server_key: "431ca85b6b474a6a18c85b5f33acd17c"
server_key_iv: "32f30b12db5239cb390bdd88063ddacf"
server_key_version: "encryptionkey@0"
I20260805 22:46:49.940011 24977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/wal/instance:
uuid: "5b3938a9110044f1b9e5d2e13240a0eb"
format_stamp: "Formatted at 2026-08-05 22:46:49 on dist-test-slave-g170"
server_key: "431ca85b6b474a6a18c85b5f33acd17c"
server_key_iv: "32f30b12db5239cb390bdd88063ddacf"
server_key_version: "encryptionkey@0"
I20260805 22:46:49.940593 24966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } attempt: 2
I20260805 22:46:49.941062 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.940899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:46625 (local address 127.0.0.1:49050)
0805 22:46:49.940974 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.941017 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260805 22:46:49.941449 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.941337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:46625 (local address 127.0.0.1:49054)
0805 22:46:49.941389 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.941416 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260805 22:46:49.941531 24977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
W20260805 22:46:49.941573 24966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.125:46625: Network error: Client connection negotiation failed: client connection to 127.23.197.125:46625: connect: Connection refused (error 111)
I20260805 22:46:49.942375 24998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:49.942552 24977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:49.942694 24977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/wal
uuid: "5b3938a9110044f1b9e5d2e13240a0eb"
format_stamp: "Formatted at 2026-08-05 22:46:49 on dist-test-slave-g170"
server_key: "431ca85b6b474a6a18c85b5f33acd17c"
server_key_iv: "32f30b12db5239cb390bdd88063ddacf"
server_key_version: "encryptionkey@0"
I20260805 22:46:49.942752 24977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:49.951696 24977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:49.953668 24977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:49.953778 24977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:49.958055 24977 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.125:46625
I20260805 22:46:49.958122 25070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.125:46625 every 8 connection(s)
I20260805 22:46:49.958549 24977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
I20260805 22:46:49.959489 25071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:49.961208 25071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:49.963073 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 24977
I20260805 22:46:49.963212 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-1/wal/instance
I20260805 22:46:49.963405 24341 external_mini_cluster.cc:1428] Setting key 69368271416d604032e271751986fb56
Aug 05 22:46:49 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:49.972122 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.961920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45694 (local address 127.23.197.126:41855)
0805 22:46:49.961996 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:49.961999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:49.962216 (+ 217us) server_negotiation.cc:407] Connection header received
0805 22:46:49.962768 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:49.962770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:49.962784 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:49.962846 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:49.963692 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.964429 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.965604 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.965821 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.965909 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:49.968605 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:49.968607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:49.968607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:49.968619 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:49.970345 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:49.970945 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:49.970946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:49.970947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:49.971012 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:49.971350 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:49.971351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:49.971351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:49.971478 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:49.971537 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:49.971937 (+ 400us) server_negotiation.cc:299] Negotiation successful
0805 22:46:49.972017 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 22:46:49.972339 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.961888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41855 (local address 127.0.0.1:45694)
0805 22:46:49.962077 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.962120 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:49.962650 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:49.962928 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:49.962931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:49.963252 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:49.963544 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:49.963550 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.964553 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.964555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:49.965497 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:49.965500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.965918 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.965919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:49.966055 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:49.966629 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:49.966638 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:49.968476 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:49.970445 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:49.970455 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:49.970458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:49.970834 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:49.971106 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:49.971107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:49.971108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:49.971267 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:49.971561 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:49.971568 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:49.971771 (+ 203us) client_negotiation.cc:769] Sending connection context
0805 22:46:49.971907 (+ 136us) client_negotiation.cc:240] Negotiation successful
0805 22:46:49.972098 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.23.197.124@KRBTEST.COM; defaulting to no policy
I20260805 22:46:49.973013 25071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.124@KRBTEST.COM" created.
Aug 05 22:46:49 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:46:49.981564 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.973345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:46625 (local address 127.0.0.1:49068)
0805 22:46:49.973442 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.973453 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:49.973500 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:49.973768 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:49.973769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:49.973911 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:49.973958 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:49.973960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.974948 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.974949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:49.975556 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:49.975559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.976061 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.976062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:49.976125 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:49.976575 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:49.976579 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:49.978008 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:49.980314 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:49.980315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:49.980315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:49.980484 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:49.980854 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:49.980855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:49.980855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:49.980918 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:49.981304 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:49.981304 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:49.981346 (+ 42us) client_negotiation.cc:769] Sending connection context
0805 22:46:49.981406 (+ 60us) client_negotiation.cc:240] Negotiation successful
0805 22:46:49.981494 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260805 22:46:49.981734 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.973437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49068 (local address 127.23.197.125:46625)
0805 22:46:49.973608 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:49.973613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:49.973639 (+ 26us) server_negotiation.cc:407] Connection header received
0805 22:46:49.973655 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:49.973657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:49.973667 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:49.973762 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:49.974044 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.974840 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.975649 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:49.975970 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:49.976063 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:49.978096 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:49.978106 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:49.978106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:49.978116 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:49.980211 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:49.980615 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:49.980619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:49.980619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:49.980690 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:49.980997 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:49.980998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:49.980998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:49.981225 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:49.981319 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:49.981446 (+ 127us) server_negotiation.cc:299] Negotiation successful
0805 22:46:49.981576 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260805 22:46:49.982615 25071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:49.982995 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.982869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34620)
0805 22:46:49.982923 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.982972 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260805 22:46:49.983173 25002 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
I20260805 22:46:49.983832 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:49.983574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34636)
0805 22:46:49.983787 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:49.983814 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
W20260805 22:46:49.984004 25071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38225: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab.
Entry for principal kudu/127.23.197.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab.
I20260805 22:46:49.996009 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.124:38225
--webserver_interface=127.23.197.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab
--principal=kudu/127.23.197.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:41855,127.23.197.125:46625,127.23.197.124:38225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:46:50.032822 24966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } attempt: 3
I20260805 22:46:50.033264 25071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } attempt: 1
I20260805 22:46:50.033748 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.033585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34638)
0805 22:46:50.033648 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.033694 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260805 22:46:50.034202 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.034070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34644)
0805 22:46:50.034129 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.034164 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260805 22:46:50.034338 25071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38225: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:46:50.041231 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.033292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:46625 (local address 127.0.0.1:49072)
0805 22:46:50.033368 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.033381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:50.033434 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:50.033790 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:50.033792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:50.034008 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:50.034102 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.034105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.034978 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.034979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.035531 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.035533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.035955 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.035956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.036038 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.036527 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:50.036533 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:50.038069 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:50.039872 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.039873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.039873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.040070 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.040311 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.040312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.040312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.040379 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.040647 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:50.040648 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:50.040876 (+ 228us) client_negotiation.cc:769] Sending connection context
0805 22:46:50.040984 (+ 108us) client_negotiation.cc:240] Negotiation successful
0805 22:46:50.041125 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260805 22:46:50.041230 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.033410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49072 (local address 127.23.197.125:46625)
0805 22:46:50.033639 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.033641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.033651 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:46:50.033678 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.033678 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.033690 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.033751 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.034243 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.034877 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.035622 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.035853 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.035931 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.038199 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.038201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.038201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.038208 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.039758 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.040172 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.040173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.040173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.040241 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.040448 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.040449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.040449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.040570 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.040645 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.041029 (+ 384us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.041124 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 22:46:50.041960 24966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:50.042361 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.042237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34650)
0805 22:46:50.042291 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.042334 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260805 22:46:50.042717 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.042621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34658)
0805 22:46:50.042667 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.042696 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260805 22:46:50.042842 24966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38225: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
I20260805 22:46:50.074013 24966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } attempt: 1
I20260805 22:46:50.074690 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.074504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34666)
0805 22:46:50.074576 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.074620 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260805 22:46:50.075201 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.075052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34670)
0805 22:46:50.075137 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.075173 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260805 22:46:50.075381 24966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38225: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
W20260805 22:46:50.089165 25079 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 22:46:50.089450 25079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:50.089497 25079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:50.089566 25079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:50.091010 25079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:50.091080 25079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:50.091110 25079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:50.091146 25079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:50.091171 25079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:50.093003 25079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:41855,127.23.197.125:46625,127.23.197.124:38225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab
--principal=kudu/127.23.197.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.124:38225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.197.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:50.093317 25079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:50.093573 25079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:50.099026 25085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:50.099025 25087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:50.099040 25084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:50.099406 25079 server_base.cc:1034] running on GCE node
I20260805 22:46:50.099669 25079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:50.100001 25079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:50.101150 25079 hybrid_clock.cc:648] HybridClock initialized: now 1785970010101130 us; error 34 us; skew 500 ppm
I20260805 22:46:50.102474 25071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } attempt: 2
I20260805 22:46:50.102914 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.102754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34686)
0805 22:46:50.102823 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.102867 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260805 22:46:50.103328 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.103227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34688)
0805 22:46:50.103263 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.103296 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260805 22:46:50.103456 25071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:38225: Network error: Client connection negotiation failed: client connection to 127.23.197.124:38225: connect: Connection refused (error 111)
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970010, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:50.105741 25079 init.cc:377] Logged in from keytab as kudu/127.23.197.124@KRBTEST.COM (short username kudu)
I20260805 22:46:50.106374 25079 webserver.cc:460] Webserver started at http://127.23.197.124:45351/ using document root <none> and password file <none>
I20260805 22:46:50.106650 25079 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:50.106712 25079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:50.106900 25079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:50.108660 25079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/data/instance:
uuid: "bba9baa05f3f416bbabf21f247307ba9"
format_stamp: "Formatted at 2026-08-05 22:46:50 on dist-test-slave-g170"
server_key: "cf735baf7502e8c2a282dd7423586236"
server_key_iv: "7f67f5897f2555304e5b3355cc836615"
server_key_version: "encryptionkey@0"
I20260805 22:46:50.109071 25079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/wal/instance:
uuid: "bba9baa05f3f416bbabf21f247307ba9"
format_stamp: "Formatted at 2026-08-05 22:46:50 on dist-test-slave-g170"
server_key: "cf735baf7502e8c2a282dd7423586236"
server_key_iv: "7f67f5897f2555304e5b3355cc836615"
server_key_version: "encryptionkey@0"
I20260805 22:46:50.112259 25079 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260805 22:46:50.113600 25097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:50.113831 25079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:50.113948 25079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/wal
uuid: "bba9baa05f3f416bbabf21f247307ba9"
format_stamp: "Formatted at 2026-08-05 22:46:50 on dist-test-slave-g170"
server_key: "cf735baf7502e8c2a282dd7423586236"
server_key_iv: "7f67f5897f2555304e5b3355cc836615"
server_key_version: "encryptionkey@0"
I20260805 22:46:50.114032 25079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:50.129969 25079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:50.132673 25079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:50.132910 25079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:50.137523 25079 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.124:38225
I20260805 22:46:50.137629 25179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.124:38225 every 8 connection(s)
I20260805 22:46:50.137979 25079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
I20260805 22:46:50.139154 25180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:50.141103 25180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:50.142834 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 25079
I20260805 22:46:50.142982 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785969725087118-24341-0/minicluster-data/master-2/wal/instance
I20260805 22:46:50.143191 24341 external_mini_cluster.cc:1428] Setting key e55971855f28c2e888a8f75e0972481c
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970010, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
I20260805 22:46:50.150569 24966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } attempt: 2
I20260805 22:46:50.152261 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.142029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.23.197.126:41855)
0805 22:46:50.142112 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.142115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.142286 (+ 171us) server_negotiation.cc:407] Connection header received
0805 22:46:50.142841 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.142843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.142859 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.142931 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.143799 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.144405 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.145831 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.146102 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.146180 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.148793 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.148796 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.148796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.148806 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.150477 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.151080 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.151081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.151082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.151153 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.151499 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.151500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.151505 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.151638 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.151722 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.152076 (+ 354us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.152155 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 22:46:50.152868 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.141906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:41855 (local address 127.0.0.1:45696)
0805 22:46:50.142139 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.142190 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:50.142715 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:50.143019 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:50.143023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:50.143402 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:50.143655 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.143662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.144509 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.144510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.145744 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.145747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.146212 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.146214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.146337 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.147010 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:50.147019 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:50.148671 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:50.150598 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.150605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.150607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.150975 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.151247 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.151248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.151248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.151408 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.151717 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:50.151725 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:50.151900 (+ 175us) client_negotiation.cc:769] Sending connection context
0805 22:46:50.152356 (+ 456us) client_negotiation.cc:240] Negotiation successful
0805 22:46:50.152543 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":122,"threads_started":1}
I20260805 22:46:50.153412 25180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:50.154373 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.143637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.23.197.124:38225)
0805 22:46:50.143839 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.143844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.143882 (+ 38us) server_negotiation.cc:407] Connection header received
0805 22:46:50.143932 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.143934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.143959 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.144047 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.144635 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.145846 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.146880 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.147283 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.147350 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.150293 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.150310 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.150311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.150318 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.152600 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.153328 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.153331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.153331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.153393 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.153607 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.153608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.153609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.153776 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.153950 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.154178 (+ 228us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.154269 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":104,"threads_started":1}
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970010, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:46:50.159945 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.150920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34702)
0805 22:46:50.151004 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.151016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:50.151071 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:50.151592 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:50.151594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:50.151754 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:50.151791 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.151793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.152582 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.152583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.153511 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.153514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.154275 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.154277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.154371 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.154869 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:50.154876 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:50.156743 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:50.158862 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.158863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.158864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.159085 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.159313 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.159314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.159315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.159379 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.159689 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:50.159690 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:50.159751 (+ 61us) client_negotiation.cc:769] Sending connection context
0805 22:46:50.159803 (+ 52us) client_negotiation.cc:240] Negotiation successful
0805 22:46:50.159880 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260805 22:46:50.160341 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.151169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.23.197.124:38225)
0805 22:46:50.151370 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.151372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.151382 (+ 10us) server_negotiation.cc:407] Connection header received
0805 22:46:50.151404 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.151405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.151417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.151568 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.151878 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.152482 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.153795 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.154161 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.154549 (+ 388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.156858 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.156859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.156860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.156868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.158773 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.159170 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.159171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.159172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.159235 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.159454 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.159455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.159455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.159592 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.160080 (+ 488us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.160127 (+ 47us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.160207 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":114,"threads_started":1}
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:50.162258 24966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519: Bootstrap starting.
I20260805 22:46:50.162599 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.153684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:46625 (local address 127.0.0.1:49080)
0805 22:46:50.153745 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.153756 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:50.153795 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:50.153949 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:50.153949 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:50.154107 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:50.154159 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.154160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.155695 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.155696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.156510 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.156513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.156978 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.156979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.157056 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.158048 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:50.158052 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:50.159356 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:50.161475 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.161476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.161477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.161688 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.161935 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.161936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.161936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.161995 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.162294 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:50.162295 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:50.162336 (+ 41us) client_negotiation.cc:769] Sending connection context
0805 22:46:50.162428 (+ 92us) client_negotiation.cc:240] Negotiation successful
0805 22:46:50.162514 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260805 22:46:50.162984 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.153757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.23.197.125:46625)
0805 22:46:50.153803 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.153805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.153816 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:46:50.153866 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.153867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.153877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.153934 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.154281 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.155564 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.156581 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.156891 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.156971 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.159455 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.159456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.159457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.159464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.161378 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.161797 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.161798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.161799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.161858 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.162071 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.162073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.162073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.162202 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.162245 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.162478 (+ 233us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.162566 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260805 22:46:50.163141 24966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:50.163476 24966 log.cc:824] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:50.163383 25180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:50.164484 24966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519: No bootstrap required, opened a new log
I20260805 22:46:50.165829 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.155886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.23.197.126:41855)
0805 22:46:50.155969 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.155971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.156325 (+ 354us) server_negotiation.cc:407] Connection header received
0805 22:46:50.156397 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.156398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.156410 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.156474 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.157136 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.157907 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.159055 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.159312 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.159397 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.162431 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.162433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.162433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.162442 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.164496 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.164972 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.164973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.164974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.165036 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.165266 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.165267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.165267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.165406 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.165564 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.165656 (+ 92us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.165745 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 22:46:50.167464 24966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } }
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970010, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
I20260805 22:46:50.167634 24966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56cb4a79fe124aefb42e58703ac35519, State: Initialized, Role: FOLLOWER
I20260805 22:46:50.167728 24966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } }
I20260805 22:46:50.168025 25188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.168251 25188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:50.168113 24966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [sys.catalog]: configured and running, proceeding with master startup.
W20260805 22:46:50.170723 25206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:50.170794 25206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:50.170992 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.164017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34710)
0805 22:46:50.164099 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.164108 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:50.164161 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:50.164274 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:50.164274 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:50.164407 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:50.164432 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.164434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.165180 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.165180 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.165894 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.165897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.166356 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.166357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.166406 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.166816 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:50.166820 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:50.167984 (+ 1164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:50.169850 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.169851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.169851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.170064 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.170261 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.170262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.170263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.170342 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.170678 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:50.170678 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:50.170715 (+ 37us) client_negotiation.cc:769] Sending connection context
0805 22:46:50.170784 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 22:46:50.170890 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260805 22:46:50.171244 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.164097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.23.197.124:38225)
0805 22:46:50.164131 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.164133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.164140 (+ 7us) server_negotiation.cc:407] Connection header received
0805 22:46:50.164205 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.164205 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.164215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.164270 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.164517 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.165090 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.165961 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.166274 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.166359 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.168073 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.168074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.168074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.168081 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.169737 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.170146 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.170148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.170148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.170202 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.170427 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.170427 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.170428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.170567 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.170652 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.171093 (+ 441us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.171169 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:46:50.174860 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9: Bootstrap starting.
I20260805 22:46:50.172192 24867 master_runner.cc:186] Error getting master registration for 127.23.197.125:46625: OK, instance_id { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" instance_seqno: 1785970009956718 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 22:46:50.175387 24867 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 22:46:50.175597 25180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:50.175824 25180 log.cc:824] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:50.176219 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.166457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.23.197.125:46625)
0805 22:46:50.166501 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.166503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.166512 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:50.166581 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.166582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.166590 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.166641 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.167346 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.168076 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.169291 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.169639 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.169692 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.172706 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.172708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.172709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.172717 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.174550 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.175011 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.175012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.175013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.175087 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.175312 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.175313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.175313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.175448 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.175601 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.176031 (+ 430us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.176121 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 22:46:50.176916 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9: No bootstrap required, opened a new log
I20260805 22:46:50.177882 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:50.178781 25180 raft_consensus.cc:348] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } }
I20260805 22:46:50.178948 25180 raft_consensus.cc:729] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bba9baa05f3f416bbabf21f247307ba9, State: Initialized, Role: FOLLOWER
I20260805 22:46:50.179042 25180 consensus_queue.cc:260] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } }
I20260805 22:46:50.179117 24928 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "56cb4a79fe124aefb42e58703ac35519"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45700
I20260805 22:46:50.179253 24928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260805 22:46:50.179317 24928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:50.179375 25180 sys_catalog.cc:564] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:50.179490 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.179634 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:50.180606 24928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } }
I20260805 22:46:50.180910 24928 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [CANDIDATE]: Term 1 election: Requested vote from peers 5b3938a9110044f1b9e5d2e13240a0eb (127.23.197.125:46625), bba9baa05f3f416bbabf21f247307ba9 (127.23.197.124:38225)
I20260805 22:46:50.181980 25140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56cb4a79fe124aefb42e58703ac35519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bba9baa05f3f416bbabf21f247307ba9"
I20260805 22:46:50.182090 25140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:50.183192 25140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56cb4a79fe124aefb42e58703ac35519 in term 1.
I20260805 22:46:50.183418 24899 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56cb4a79fe124aefb42e58703ac35519, bba9baa05f3f416bbabf21f247307ba9; no voters:
I20260805 22:46:50.183550 25188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:50.183717 25188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [term 1 LEADER]: Becoming Leader. State: Replica: 56cb4a79fe124aefb42e58703ac35519, State: Running, Role: LEADER
I20260805 22:46:50.183799 25188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } }
I20260805 22:46:50.185074 25237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56cb4a79fe124aefb42e58703ac35519. Latest consensus state: current_term: 1 leader_uuid: "56cb4a79fe124aefb42e58703ac35519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.185209 25237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [sys.catalog]: This master's current role is: LEADER
W20260805 22:46:50.185163 25226 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:50.185319 25226 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:50.185995 25079 master_runner.cc:186] Error getting master registration for 127.23.197.125:46625: OK, instance_id { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" instance_seqno: 1785970009956718 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 22:46:50.188190 25079 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 22:46:50.192113 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.182124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.23.197.126:41855)
0805 22:46:50.182210 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.182212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.182558 (+ 346us) server_negotiation.cc:407] Connection header received
0805 22:46:50.182675 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.182676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.182687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.182755 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.184057 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.184983 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.186721 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.187052 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.187098 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.188867 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.188868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.188869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.188876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.190654 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.191048 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.191050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.191050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.191112 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.191468 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.191469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.191469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.191618 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.191732 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.191927 (+ 195us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.191999 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 22:46:50.192930 25240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:50.193212 25240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:50.194013 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.182065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.23.197.125:46625)
0805 22:46:50.182175 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.182177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.185450 (+ 3273us) server_negotiation.cc:407] Connection header received
0805 22:46:50.185545 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.185546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.185555 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.185583 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.186350 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.187046 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.188010 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.188299 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.188345 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.190061 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.190062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.190063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.190070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.192020 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.193034 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.193035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.193035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.193103 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.193382 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.193383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.193383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.193524 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.193617 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.193806 (+ 189us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.193892 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260805 22:46:50.194715 25071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } attempt: 3
I20260805 22:46:50.195359 25140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [term 1 FOLLOWER]: Refusing update from remote peer 56cb4a79fe124aefb42e58703ac35519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:50.195684 25188 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:50.197151 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56cb4a79fe124aefb42e58703ac35519. Latest consensus state: current_term: 1 leader_uuid: "56cb4a79fe124aefb42e58703ac35519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.197269 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:50.198287 25237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56cb4a79fe124aefb42e58703ac35519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.198413 25237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:50.198482 25242 mvcc.cc:205] Tried to move back new op lower bound from 7315333161756049408 to 7315333161713397760. Current Snapshot: MvccSnapshot[applied={T|T < 7315333161756049408}]
I20260805 22:46:50.198531 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56cb4a79fe124aefb42e58703ac35519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.198625 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:50.199980 25245 mvcc.cc:205] Tried to move back new op lower bound from 7315333161756049408 to 7315333161713397760. Current Snapshot: MvccSnapshot[applied={T|T < 7315333161756049408}]
Aug 05 22:46:50 dist-test-slave-g170 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970009, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
I20260805 22:46:50.202608 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.183387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34716 (local address 127.23.197.124:38225)
0805 22:46:50.183438 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.183440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.192079 (+ 8639us) server_negotiation.cc:407] Connection header received
0805 22:46:50.192467 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.192468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.192479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.192512 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.193207 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.193906 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.195189 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.195538 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.195605 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.199183 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.199184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.199185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.199190 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.201175 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.201625 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.201627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.201627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.201688 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.201937 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.201938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.201938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.202114 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.202255 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.202360 (+ 105us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.202509 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260805 22:46:50.203714 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.194956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:38225 (local address 127.0.0.1:34730)
0805 22:46:50.195088 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:50.195098 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:50.195135 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:50.195686 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:50.195686 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:50.195872 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:50.195908 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.195909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.196955 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.196956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.197727 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:50.197729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.198398 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.198399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:50.198456 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.198911 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:50.198916 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:50.200540 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:50.202582 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.202583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.202584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.202788 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.203033 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:50.203034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:50.203034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:50.203110 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:50.203407 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:50.203408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:50.203446 (+ 38us) client_negotiation.cc:769] Sending connection context
0805 22:46:50.203510 (+ 64us) client_negotiation.cc:240] Negotiation successful
0805 22:46:50.203608 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260805 22:46:50.203745 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:50.195492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34730 (local address 127.23.197.124:38225)
0805 22:46:50.195549 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.195551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.195559 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:50.195577 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.195578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.195588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.195641 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.195992 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.196859 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.197825 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:50.198263 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:50.198344 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:50.200632 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:50.200634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:50.200634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:50.200641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:50.202520 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.202875 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.202876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.202877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.202954 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:50.203189 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:50.203190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:50.203191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:50.203332 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:50.203402 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:50.203559 (+ 157us) server_negotiation.cc:299] Negotiation successful
0805 22:46:50.203652 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 22:46:50.205330 25071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb: Bootstrap starting.
I20260805 22:46:50.206121 25071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:50.206444 25071 log.cc:824] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:50.207356 25071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb: No bootstrap required, opened a new log
I20260805 22:46:50.209549 25071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } }
I20260805 22:46:50.209786 25071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3938a9110044f1b9e5d2e13240a0eb, State: Initialized, Role: FOLLOWER
I20260805 22:46:50.209885 25071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } }
I20260805 22:46:50.210172 25253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.210244 25071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:50.210354 25041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:50.210371 25042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56cb4a79fe124aefb42e58703ac35519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5b3938a9110044f1b9e5d2e13240a0eb"
I20260805 22:46:50.211237 25041 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [term 1 FOLLOWER]: Refusing update from remote peer 56cb4a79fe124aefb42e58703ac35519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:50.211558 25237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:50.210263 25253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:50.212464 25042 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56cb4a79fe124aefb42e58703ac35519 in term 1.
I20260805 22:46:50.213236 25240 catalog_manager.cc:1347] Generated new cluster ID: 83b995e63c6d4a4cad3c7f6a329a7bfe
I20260805 22:46:50.213297 25240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:50.214021 25253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56cb4a79fe124aefb42e58703ac35519. Latest consensus state: current_term: 1 leader_uuid: "56cb4a79fe124aefb42e58703ac35519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.214098 25253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:50.214133 25253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56cb4a79fe124aefb42e58703ac35519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.214184 25253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:50.215032 25270 mvcc.cc:205] Tried to move back new op lower bound from 7315333161756049408 to 7315333161713397760. Current Snapshot: MvccSnapshot[applied={T|T < 7315333161756049408}]
W20260805 22:46:50.215525 25275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:50.215591 25275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:50.219211 25237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56cb4a79fe124aefb42e58703ac35519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cb4a79fe124aefb42e58703ac35519" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 41855 } } peers { permanent_uuid: "5b3938a9110044f1b9e5d2e13240a0eb" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 46625 } } peers { permanent_uuid: "bba9baa05f3f416bbabf21f247307ba9" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 38225 } } }
I20260805 22:46:50.219328 25237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:50.220640 25240 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:50.221427 25240 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:50.228642 25240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56cb4a79fe124aefb42e58703ac35519: Generated new TSK 0
I20260805 22:46:50.228868 25240 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 22:46:50.265749 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.260804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.23.197.125:46625)
0805 22:46:50.260857 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.260860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.264987 (+ 4127us) server_negotiation.cc:407] Connection header received
0805 22:46:50.265109 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.265110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.265132 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.265168 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.265669 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49114: BlockingRecv error: recv got EOF from 127.0.0.1:49114 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260805 22:46:50.266597 25184 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.261293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.23.197.124:38225)
0805 22:46:50.261337 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.261339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.265983 (+ 4644us) server_negotiation.cc:407] Connection header received
0805 22:46:50.266094 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.266096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.266106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.266151 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.266532 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34740: BlockingRecv error: recv got EOF from 127.0.0.1:34740 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260805 22:46:50.267421 24972 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.261650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.23.197.126:41855)
0805 22:46:50.261694 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.261696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.266851 (+ 5155us) server_negotiation.cc:407] Connection header received
0805 22:46:50.266960 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.266962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.266975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.267009 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.267352 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45718: BlockingRecv error: recv got EOF from 127.0.0.1:45718 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260805 22:46:50.270354 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.269393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.23.197.125:46625)
0805 22:46:50.269432 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.269434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.269704 (+ 270us) server_negotiation.cc:407] Connection header received
0805 22:46:50.269851 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.269852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.269873 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.269906 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.270300 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49122: BlockingRecv error: recv got EOF from 127.0.0.1:49122 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260805 22:46:50.275938 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.274979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49124 (local address 127.23.197.125:46625)
0805 22:46:50.275020 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.275022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.275346 (+ 324us) server_negotiation.cc:407] Connection header received
0805 22:46:50.275464 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.275465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.275474 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.275505 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.275881 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49124: BlockingRecv error: recv got EOF from 127.0.0.1:49124 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260805 22:46:50.279997 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.279168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49138 (local address 127.23.197.125:46625)
0805 22:46:50.279207 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.279209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.279464 (+ 255us) server_negotiation.cc:407] Connection header received
0805 22:46:50.279567 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.279568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.279578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.279611 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.279950 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49138: BlockingRecv error: recv got EOF from 127.0.0.1:49138 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260805 22:46:50.286182 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.285320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.23.197.125:46625)
0805 22:46:50.285358 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.285360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.285622 (+ 262us) server_negotiation.cc:407] Connection header received
0805 22:46:50.285731 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.285732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.285742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.285776 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.286138 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49148: BlockingRecv error: recv got EOF from 127.0.0.1:49148 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260805 22:46:50.296538 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.295568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.23.197.125:46625)
0805 22:46:50.295615 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.295617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.295915 (+ 298us) server_negotiation.cc:407] Connection header received
0805 22:46:50.296031 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.296032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.296043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.296078 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.296478 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49164: BlockingRecv error: recv got EOF from 127.0.0.1:49164 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260805 22:46:50.315187 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.314097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49178 (local address 127.23.197.125:46625)
0805 22:46:50.314160 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.314163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.314503 (+ 340us) server_negotiation.cc:407] Connection header received
0805 22:46:50.314625 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.314626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.314639 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.314676 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.315111 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49178: BlockingRecv error: recv got EOF from 127.0.0.1:49178 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260805 22:46:50.349887 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.348712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.23.197.125:46625)
0805 22:46:50.348797 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.348799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.349186 (+ 387us) server_negotiation.cc:407] Connection header received
0805 22:46:50.349314 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.349316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.349330 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.349364 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.349820 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49188: BlockingRecv error: recv got EOF from 127.0.0.1:49188 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260805 22:46:50.416172 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.415410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.23.197.125:46625)
0805 22:46:50.415462 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.415465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.415596 (+ 131us) server_negotiation.cc:407] Connection header received
0805 22:46:50.415674 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.415676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.415694 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.415750 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.416104 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49194: BlockingRecv error: recv got EOF from 127.0.0.1:49194 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260805 22:46:50.546455 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.545565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.23.197.125:46625)
0805 22:46:50.545657 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.545660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.545839 (+ 179us) server_negotiation.cc:407] Connection header received
0805 22:46:50.545924 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.545925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.545939 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.546021 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.546389 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49206: BlockingRecv error: recv got EOF from 127.0.0.1:49206 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260805 22:46:50.804740 25076 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:50.803912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49210 (local address 127.23.197.125:46625)
0805 22:46:50.803981 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:50.803983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:50.804058 (+ 75us) server_negotiation.cc:407] Connection header received
0805 22:46:50.804156 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:50.804157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:50.804170 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:50.804254 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:50.804655 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49210: BlockingRecv error: recv got EOF from 127.0.0.1:49210 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 22:46:51.186200 25226 catalog_manager.cc:1259] Loaded cluster ID: 83b995e63c6d4a4cad3c7f6a329a7bfe
I20260805 22:46:51.186317 25226 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9: loading cluster ID for follower catalog manager: success
I20260805 22:46:51.187183 25226 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9: acquiring CA information for follower catalog manager: success
I20260805 22:46:51.187510 25226 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bba9baa05f3f416bbabf21f247307ba9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 22:46:51.215988 25275 catalog_manager.cc:1259] Loaded cluster ID: 83b995e63c6d4a4cad3c7f6a329a7bfe
I20260805 22:46:51.216068 25275 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb: loading cluster ID for follower catalog manager: success
I20260805 22:46:51.217159 25275 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb: acquiring CA information for follower catalog manager: success
I20260805 22:46:51.217520 25275 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b3938a9110044f1b9e5d2e13240a0eb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 22:46:51.321316 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:51.318263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49226 (local address 127.23.197.125:46625)
0805 22:46:51.318427 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:51.318429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:51.318566 (+ 137us) server_negotiation.cc:407] Connection header received
0805 22:46:51.318635 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:51.318636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:51.318735 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:51.318835 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:51.319007 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.319569 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.320398 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.320612 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.320681 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:51.320947 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:51.321066 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:51.321126 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:51.321175 (+ 49us) server_negotiation.cc:299] Negotiation successful
0805 22:46:51.321197 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260805 22:46:51.326323 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:51.323521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.23.197.124:38225)
0805 22:46:51.323669 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:51.323670 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:51.323679 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:51.323719 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:51.323720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:51.323756 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:51.323835 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:51.323965 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.324511 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.325394 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.325597 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.325671 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:51.325871 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:51.325985 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:51.326068 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:51.326145 (+ 77us) server_negotiation.cc:299] Negotiation successful
0805 22:46:51.326166 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260805 22:46:51.331647 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:51.328274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.23.197.125:46625)
0805 22:46:51.328319 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:51.328321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:51.328563 (+ 242us) server_negotiation.cc:407] Connection header received
0805 22:46:51.328669 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:51.328670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:51.328681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:51.328710 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:51.328899 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.329670 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.330748 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.331047 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.331107 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:51.331367 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:51.331404 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:51.331463 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:51.331541 (+ 78us) server_negotiation.cc:299] Negotiation successful
0805 22:46:51.331566 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:46:51.331770 25397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:51.328315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.23.197.126:41855)
0805 22:46:51.328423 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:51.328425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:51.328437 (+ 12us) server_negotiation.cc:407] Connection header received
0805 22:46:51.328514 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:51.328515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:51.328572 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:51.328696 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:51.328829 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.329449 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.330648 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.331024 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.331107 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:51.331341 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:51.331430 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:51.331510 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:51.331680 (+ 170us) server_negotiation.cc:299] Negotiation successful
0805 22:46:51.331705 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":55,"threads_started":1}
I20260805 22:46:51.332808 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:51.328554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.23.197.124:38225)
0805 22:46:51.328593 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:51.328595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:51.328606 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:46:51.328631 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:51.328632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:51.328643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:51.328698 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:51.328971 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.329677 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.331948 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:51.332272 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:51.332320 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:51.332504 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:51.332538 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:51.332595 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:51.332678 (+ 83us) server_negotiation.cc:299] Negotiation successful
0805 22:46:51.332698 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260805 22:46:51.333802 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24867
I20260805 22:46:51.347313 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 24977
I20260805 22:46:51.359509 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 25079
2026-08-05T22:46:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3092 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 22:46:51 dist-test-slave-g170 krb5kdc[25402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 22:46:51 dist-test-slave-g170 krb5kdc[25402](info): set up 2 sockets
Aug 05 22:46:51 dist-test-slave-g170 krb5kdc[25402](info): commencing operation
krb5kdc: starting...
W20260805 22:46:52.621500 24341 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/test-user.keytab.
Aug 05 22:46:52 dist-test-slave-g170 krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T22:46:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T22:46:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.197.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
Entry for principal kudu/127.23.197.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab.
I20260805 22:46:52.753047 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:42145
--webserver_interface=127.23.197.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:42145,127.23.197.125:41613,127.23.197.124:44413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 22:46:52.843775 25419 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 22:46:52.844026 25419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:52.844048 25419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:52.844070 25419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:52.845407 25419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:52.845458 25419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:52.845474 25419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:52.845489 25419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:52.845504 25419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:52.847340 25419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:42145,127.23.197.125:41613,127.23.197.124:44413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.126.keytab
--principal=kudu/127.23.197.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.126:42145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.197.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:52.847558 25419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:52.847770 25419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:52.853240 25431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:52.853202 25427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:52.853258 25419 server_base.cc:1034] running on GCE node
W20260805 22:46:52.853220 25434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:52.853812 25419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:52.854168 25419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:52.855342 25419 hybrid_clock.cc:648] HybridClock initialized: now 1785970012855334 us; error 52 us; skew 500 ppm
Aug 05 22:46:52 dist-test-slave-g170 krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970012, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:52.858071 25419 init.cc:377] Logged in from keytab as kudu/127.23.197.126@KRBTEST.COM (short username kudu)
I20260805 22:46:52.858531 25419 webserver.cc:460] Webserver started at http://127.23.197.126:43607/ using document root <none> and password file <none>
I20260805 22:46:52.858842 25419 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:52.858914 25419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:52.859076 25419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:52.860204 25419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/data/instance:
uuid: "ef3b68e7668d46a599a100023c3585e1"
format_stamp: "Formatted at 2026-08-05 22:46:52 on dist-test-slave-g170"
server_key: "ad9774529fb37d8341e909dade72b4e0"
server_key_iv: "a30070f9c0588788d1b7f87202dead2f"
server_key_version: "encryptionkey@0"
I20260805 22:46:52.860594 25419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/wal/instance:
uuid: "ef3b68e7668d46a599a100023c3585e1"
format_stamp: "Formatted at 2026-08-05 22:46:52 on dist-test-slave-g170"
server_key: "ad9774529fb37d8341e909dade72b4e0"
server_key_iv: "a30070f9c0588788d1b7f87202dead2f"
server_key_version: "encryptionkey@0"
I20260805 22:46:52.862078 25419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:52.862893 25445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:52.863077 25419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:52.863179 25419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/wal
uuid: "ef3b68e7668d46a599a100023c3585e1"
format_stamp: "Formatted at 2026-08-05 22:46:52 on dist-test-slave-g170"
server_key: "ad9774529fb37d8341e909dade72b4e0"
server_key_iv: "a30070f9c0588788d1b7f87202dead2f"
server_key_version: "encryptionkey@0"
I20260805 22:46:52.863265 25419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:52.880748 25419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:52.882997 25419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:52.883193 25419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:52.887418 25419 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.126:42145
I20260805 22:46:52.887463 25528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.126:42145 every 8 connection(s)
I20260805 22:46:52.887879 25419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/data/info.pb
I20260805 22:46:52.888934 25529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:52.889380 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 25419
I20260805 22:46:52.889497 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-0/wal/instance
I20260805 22:46:52.889765 24341 external_mini_cluster.cc:1428] Setting key 87bd5e78b59957a96bc323f0f4589eca
I20260805 22:46:52.891727 25529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.197.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.125@KRBTEST.COM" created.
Aug 05 22:46:52 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970012, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:52.907732 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:52.892713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.23.197.126:42145)
0805 22:46:52.892953 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:52.892965 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:52.893146 (+ 181us) server_negotiation.cc:407] Connection header received
0805 22:46:52.893816 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:52.893818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:52.893837 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:52.893942 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:52.894647 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:52.896243 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:52.897336 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:52.897959 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:52.898099 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:52.903407 (+ 5308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:52.903415 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:52.903416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:52.903423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:52.905535 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:52.906137 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:52.906140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:52.906141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:52.906233 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:52.906530 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:52.906530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:52.906531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:52.906772 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:52.906845 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:52.907150 (+ 305us) server_negotiation.cc:299] Negotiation successful
0805 22:46:52.907461 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":196,"server-negotiator.queue_time_us":176,"thread_start_us":138,"threads_started":1}
I20260805 22:46:52.907738 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:52.892713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:42145 (local address 127.0.0.1:50364)
0805 22:46:52.892967 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:52.893011 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:52.893619 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:52.893946 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:52.893948 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:52.894282 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:52.894484 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:52.894491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:52.896380 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:52.896381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:52.897241 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:52.897244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:52.898077 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:52.898078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:52.898144 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:52.898773 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:52.898783 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:52.903288 (+ 4505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:52.905652 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:52.905660 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:52.905663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:52.905985 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:52.906311 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:52.906312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:52.906313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:52.906450 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:52.906856 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:52.906865 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:52.906953 (+ 88us) client_negotiation.cc:769] Sending connection context
0805 22:46:52.907060 (+ 107us) client_negotiation.cc:240] Negotiation successful
0805 22:46:52.907346 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":137,"threads_started":1}
I20260805 22:46:52.908838 25529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:52.909272 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:52.909102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:41613 (local address 127.0.0.1:54808)
0805 22:46:52.909164 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:52.909234 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260805 22:46:52.909435 25446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
I20260805 22:46:52.909981 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:52.909857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:41613 (local address 127.0.0.1:54816)
0805 22:46:52.909911 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:52.909948 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260805 22:46:52.910204 25529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.125:41613: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab.
Entry for principal kudu/127.23.197.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab.
I20260805 22:46:52.920275 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.125:41613
--webserver_interface=127.23.197.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab
--principal=kudu/127.23.197.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:42145,127.23.197.125:41613,127.23.197.124:44413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:46:52.959491 25529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } attempt: 1
I20260805 22:46:52.960067 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:52.959907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:41613 (local address 127.0.0.1:54822)
0805 22:46:52.959985 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:52.960032 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260805 22:46:52.960469 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:52.960360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:41613 (local address 127.0.0.1:54834)
0805 22:46:52.960412 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:52.960446 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260805 22:46:52.960563 25529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.125:41613: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
W20260805 22:46:53.006654 25540 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 22:46:53.006903 25540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:53.006933 25540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:53.006978 25540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:53.008172 25540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:53.008240 25540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:53.008263 25540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:53.008283 25540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:53.008309 25540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:53.009980 25540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:42145,127.23.197.125:41613,127.23.197.124:44413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.125.keytab
--principal=kudu/127.23.197.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.125:41613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.197.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:53.010309 25540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:53.010563 25540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:53.015847 25550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:53.015859 25546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:53.015920 25540 server_base.cc:1034] running on GCE node
W20260805 22:46:53.015869 25547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:53.016302 25540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:53.016680 25540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:53.017869 25540 hybrid_clock.cc:648] HybridClock initialized: now 1785970013017871 us; error 67 us; skew 500 ppm
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970013, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:53.020912 25540 init.cc:377] Logged in from keytab as kudu/127.23.197.125@KRBTEST.COM (short username kudu)
I20260805 22:46:53.021440 25540 webserver.cc:460] Webserver started at http://127.23.197.125:43221/ using document root <none> and password file <none>
I20260805 22:46:53.021703 25540 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:53.021782 25540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:53.021960 25540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:53.023202 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/data/instance:
uuid: "515b617d6c7e4e4ba160b3b162cd57e6"
format_stamp: "Formatted at 2026-08-05 22:46:53 on dist-test-slave-g170"
server_key: "407a5032cefe42bc9a9dabeebf7a25da"
server_key_iv: "fa5146ea6b82df781b8d42c59350db9b"
server_key_version: "encryptionkey@0"
I20260805 22:46:53.023617 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/wal/instance:
uuid: "515b617d6c7e4e4ba160b3b162cd57e6"
format_stamp: "Formatted at 2026-08-05 22:46:53 on dist-test-slave-g170"
server_key: "407a5032cefe42bc9a9dabeebf7a25da"
server_key_iv: "fa5146ea6b82df781b8d42c59350db9b"
server_key_version: "encryptionkey@0"
I20260805 22:46:53.025278 25540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260805 22:46:53.026149 25560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:53.026346 25540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 22:46:53.026465 25540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/wal
uuid: "515b617d6c7e4e4ba160b3b162cd57e6"
format_stamp: "Formatted at 2026-08-05 22:46:53 on dist-test-slave-g170"
server_key: "407a5032cefe42bc9a9dabeebf7a25da"
server_key_iv: "fa5146ea6b82df781b8d42c59350db9b"
server_key_version: "encryptionkey@0"
I20260805 22:46:53.026551 25540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:53.028735 25529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } attempt: 2
I20260805 22:46:53.029285 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.029118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:41613 (local address 127.0.0.1:54842)
0805 22:46:53.029197 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.029245 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260805 22:46:53.029758 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.029635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:41613 (local address 127.0.0.1:54854)
0805 22:46:53.029696 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.029730 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260805 22:46:53.029870 25529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.125:41613: Network error: Client connection negotiation failed: client connection to 127.23.197.125:41613: connect: Connection refused (error 111)
I20260805 22:46:53.043052 25540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:53.045396 25540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:53.045526 25540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:53.050315 25540 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.125:41613
I20260805 22:46:53.050372 25640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.125:41613 every 8 connection(s)
I20260805 22:46:53.050747 25540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/data/info.pb
I20260805 22:46:53.051754 25641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:53.053381 25641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:53.056455 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 25540
I20260805 22:46:53.056572 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-1/wal/instance
I20260805 22:46:53.056843 24341 external_mini_cluster.cc:1428] Setting key 6a507a18e4d46896b0b781c495500ff0
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970013, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:53.064291 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.054079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50366 (local address 127.23.197.126:42145)
0805 22:46:53.054161 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.054164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.054399 (+ 235us) server_negotiation.cc:407] Connection header received
0805 22:46:53.054871 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.054873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.054886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.054947 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.055547 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.056112 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.057834 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.058045 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.058111 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.060710 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.060712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.060712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.060721 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.062604 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.063110 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.063112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.063112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.063185 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.063465 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.063467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.063467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.063597 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.063764 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.064079 (+ 315us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.064177 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 22:46:53.064502 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.054065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:42145 (local address 127.0.0.1:50366)
0805 22:46:53.054266 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.054307 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:53.054764 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:53.055022 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:53.055026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:53.055275 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:53.055464 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.055467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.056284 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.056285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.057709 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.057713 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.058149 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.058150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.058311 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.059081 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:53.059095 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:53.060595 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:53.062738 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.062747 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.062749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.063004 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.063260 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.063261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.063262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.063386 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.063756 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:53.063765 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:53.063925 (+ 160us) client_negotiation.cc:769] Sending connection context
0805 22:46:53.064042 (+ 117us) client_negotiation.cc:240] Negotiation successful
0805 22:46:53.064227 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":125,"threads_started":1}
I20260805 22:46:53.065315 25641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.197.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.197.124@KRBTEST.COM" created.
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970013, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:46:53.074608 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.065664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:41613 (local address 127.0.0.1:54866)
0805 22:46:53.065845 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.065857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:53.065919 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:53.066152 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:53.066153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:53.066286 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:53.066340 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.066341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.067559 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.067560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.068344 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.068347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.069164 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.069165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.069227 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.069743 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:53.069749 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:53.071026 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:53.073396 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.073397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.073398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.073671 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.073923 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.073923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.073924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.073983 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.074348 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:53.074349 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:53.074387 (+ 38us) client_negotiation.cc:769] Sending connection context
0805 22:46:53.074444 (+ 57us) client_negotiation.cc:240] Negotiation successful
0805 22:46:53.074533 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260805 22:46:53.074782 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.065700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54866 (local address 127.23.197.125:41613)
0805 22:46:53.065860 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.065865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.065917 (+ 52us) server_negotiation.cc:407] Connection header received
0805 22:46:53.066009 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.066012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.066025 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.066140 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.066449 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.067458 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.068445 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.069045 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.069148 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.071150 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.071158 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.071158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.071165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.073308 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.073771 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.073775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.073775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.073845 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.074052 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.074053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.074053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.074270 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.074371 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.074485 (+ 114us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.074623 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260805 22:46:53.075503 25641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:53.075932 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.075798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41340)
0805 22:46:53.075856 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.075912 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260805 22:46:53.076112 25563 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
I20260805 22:46:53.076603 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.076506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41356)
0805 22:46:53.076558 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.076585 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260805 22:46:53.076788 25641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:44413: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.197.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab.
Entry for principal kudu/127.23.197.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab.
I20260805 22:46:53.088194 24341 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHuFw91/build/release/bin/kudu
/tmp/dist-test-taskHuFw91/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.124:44413
--webserver_interface=127.23.197.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab
--principal=kudu/127.23.197.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.197.84:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.197.126:42145,127.23.197.125:41613,127.23.197.124:44413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 22:46:53.121016 25529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } attempt: 3
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970012, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:46:53.126060 25641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } attempt: 1
I20260805 22:46:53.126490 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.126331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41366)
0805 22:46:53.126401 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.126447 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260805 22:46:53.127043 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.126910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41382)
0805 22:46:53.126974 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.127006 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260805 22:46:53.127336 25641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:44413: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
I20260805 22:46:53.129287 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.121504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54870 (local address 127.23.197.125:41613)
0805 22:46:53.121576 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.121578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.121589 (+ 11us) server_negotiation.cc:407] Connection header received
0805 22:46:53.121642 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.121643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.121655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.121731 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.122105 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.122630 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.123378 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.123610 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.123693 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.125938 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.125939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.125940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.125948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.127936 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.128383 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.128384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.128384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.128437 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.128690 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.128691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.128692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.128908 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.128983 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.129157 (+ 174us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.129231 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 22:46:53.129364 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.121419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:41613 (local address 127.0.0.1:54870)
0805 22:46:53.121492 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.121503 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:53.121573 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:53.121780 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:53.121781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:53.121947 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:53.122015 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.122017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.122737 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.122739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.123290 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.123292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.123690 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.123691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.123776 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.124259 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:53.124265 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:53.125829 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:53.128029 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.128030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.128030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.128292 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.128503 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.128504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.128504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.128582 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.128986 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:53.128988 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:53.129047 (+ 59us) client_negotiation.cc:769] Sending connection context
0805 22:46:53.129138 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 22:46:53.129258 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260805 22:46:53.130028 25529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } has no permanent_uuid. Determining permanent_uuid...
I20260805 22:46:53.130409 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.130291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41386)
0805 22:46:53.130348 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.130383 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260805 22:46:53.130849 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.130738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41396)
0805 22:46:53.130786 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.130817 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260805 22:46:53.130973 25529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:44413: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
I20260805 22:46:53.162158 25529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } attempt: 1
I20260805 22:46:53.162724 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.162569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41404)
0805 22:46:53.162655 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.162693 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260805 22:46:53.163170 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.163001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41408)
0805 22:46:53.163075 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.163128 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260805 22:46:53.163334 25529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:44413: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
W20260805 22:46:53.181418 25653 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 22:46:53.181646 25653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 22:46:53.181663 25653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 22:46:53.181685 25653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 22:46:53.182821 25653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 22:46:53.182855 25653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 22:46:53.182870 25653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 22:46:53.182883 25653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 22:46:53.182894 25653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 22:46:53.184335 25653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.197.84:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.197.126:42145,127.23.197.125:41613,127.23.197.124:44413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/krb5kdc/kudu_127.23.197.124.keytab
--principal=kudu/127.23.197.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.197.124:44413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.197.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 05 Aug 2026 21:53:01 UTC on ccycloud.quasar-phufqb.root.comops.site
build id 4569240
I20260805 22:46:53.184544 25653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 22:46:53.184725 25653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 22:46:53.189746 25660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:53.189743 25664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 22:46:53.189738 25662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 22:46:53.190012 25653 server_base.cc:1034] running on GCE node
I20260805 22:46:53.190259 25653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 22:46:53.190552 25653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 22:46:53.191723 25653 hybrid_clock.cc:648] HybridClock initialized: now 1785970013191708 us; error 55 us; skew 500 ppm
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970013, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 22:46:53.194337 25653 init.cc:377] Logged in from keytab as kudu/127.23.197.124@KRBTEST.COM (short username kudu)
I20260805 22:46:53.194799 25653 webserver.cc:460] Webserver started at http://127.23.197.124:46145/ using document root <none> and password file <none>
I20260805 22:46:53.195000 25653 fs_manager.cc:359] Metadata directory not provided
I20260805 22:46:53.195043 25653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 22:46:53.195133 25653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 22:46:53.195505 25641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } attempt: 2
I20260805 22:46:53.195993 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.195837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41422)
0805 22:46:53.195911 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.195953 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260805 22:46:53.196345 25653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/data/instance:
uuid: "258bb47e936343c9b14a4df3b38556ee"
format_stamp: "Formatted at 2026-08-05 22:46:53 on dist-test-slave-g170"
server_key: "426443d695e01028c023652af613c178"
server_key_iv: "45c00dee755e77fae292d235ce37dd43"
server_key_version: "encryptionkey@0"
I20260805 22:46:53.196523 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.196405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41432)
0805 22:46:53.196461 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.196495 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260805 22:46:53.196743 25641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.197.124:44413: Network error: Client connection negotiation failed: client connection to 127.23.197.124:44413: connect: Connection refused (error 111)
I20260805 22:46:53.196751 25653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/wal/instance:
uuid: "258bb47e936343c9b14a4df3b38556ee"
format_stamp: "Formatted at 2026-08-05 22:46:53 on dist-test-slave-g170"
server_key: "426443d695e01028c023652af613c178"
server_key_iv: "45c00dee755e77fae292d235ce37dd43"
server_key_version: "encryptionkey@0"
I20260805 22:46:53.198342 25653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:53.199288 25672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 22:46:53.199476 25653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 22:46:53.199579 25653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/data,/tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/wal
uuid: "258bb47e936343c9b14a4df3b38556ee"
format_stamp: "Formatted at 2026-08-05 22:46:53 on dist-test-slave-g170"
server_key: "426443d695e01028c023652af613c178"
server_key_iv: "45c00dee755e77fae292d235ce37dd43"
server_key_version: "encryptionkey@0"
I20260805 22:46:53.199667 25653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 22:46:53.218569 25653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 22:46:53.221065 25653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 22:46:53.221244 25653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 22:46:53.225502 25653 rpc_server.cc:307] RPC server started. Bound to: 127.23.197.124:44413
I20260805 22:46:53.225541 25743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.197.124:44413 every 8 connection(s)
I20260805 22:46:53.225926 25653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/data/info.pb
I20260805 22:46:53.226814 25745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 22:46:53.228430 25745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970013, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:53.234576 24341 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHuFw91/build/release/bin/kudu as pid 25653
I20260805 22:46:53.234696 24341 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHuFw91/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785969725087118-24341-0/minicluster-data/master-2/wal/instance
I20260805 22:46:53.234889 24341 external_mini_cluster.cc:1428] Setting key 684e69fcbfca3a02ea094f00dc39eb52
I20260805 22:46:53.238497 25529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } attempt: 2
I20260805 22:46:53.238615 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.229221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50372 (local address 127.23.197.126:42145)
0805 22:46:53.229295 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.229297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.229504 (+ 207us) server_negotiation.cc:407] Connection header received
0805 22:46:53.230076 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.230077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.230088 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.230164 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.230889 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.231378 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.232411 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.232601 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.232667 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.235009 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.235011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.235011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.235019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.236935 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.237363 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.237365 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.237365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.237435 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.237790 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.237792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.237792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.237912 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.237975 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.238340 (+ 365us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.238411 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 22:46:53.238768 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.229183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.126:42145 (local address 127.0.0.1:50372)
0805 22:46:53.229365 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.229408 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:53.229955 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:53.230247 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:53.230251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:53.230574 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:53.230778 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.230783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.231470 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.231470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.232309 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.232312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.232692 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.232694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.232854 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.233328 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:53.233337 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:53.234892 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:53.237037 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.237044 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.237045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.237282 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.237531 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.237533 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.237533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.237686 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.237991 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:53.237998 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:53.238179 (+ 181us) client_negotiation.cc:769] Sending connection context
0805 22:46:53.238296 (+ 117us) client_negotiation.cc:240] Negotiation successful
0805 22:46:53.238484 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
I20260805 22:46:53.240272 25745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970012, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.126@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970013, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:46:53.246526 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.235335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.23.197.124:44413)
0805 22:46:53.235533 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.235538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.235574 (+ 36us) server_negotiation.cc:407] Connection header received
0805 22:46:53.235640 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.235643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.235659 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.235740 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.236385 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.237392 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.238880 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.239369 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.239454 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.242399 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.242419 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.242419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.242426 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.244997 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.245455 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.245459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.245460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.245530 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.245778 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.245779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.245780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.246014 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.246197 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.246256 (+ 59us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.246391 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260805 22:46:53.248013 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.238782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41446)
0805 22:46:53.238977 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.238988 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:53.239027 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:53.239253 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:53.239254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:53.239400 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:53.239460 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.239462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.240366 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.240368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.241237 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.241240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.241734 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.241735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.241795 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.242177 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:53.242182 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:53.243697 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:53.246891 (+ 3194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.246892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.246893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.247118 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.247346 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.247347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.247347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.247415 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.247700 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:53.247701 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:53.247750 (+ 49us) client_negotiation.cc:769] Sending connection context
0805 22:46:53.247801 (+ 51us) client_negotiation.cc:240] Negotiation successful
0805 22:46:53.247901 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260805 22:46:53.248421 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.238954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.23.197.124:44413)
0805 22:46:53.239134 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.239136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.239145 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:53.239165 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.239166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.239177 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.239245 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.239555 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.240278 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.241332 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.241647 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.241733 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.243826 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.243827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.243828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.243834 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.246828 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.247203 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.247203 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.247204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.247266 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.247492 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.247493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.247493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.247618 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.248148 (+ 530us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.248193 (+ 45us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.248280 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":121,"threads_started":1}
I20260805 22:46:53.249671 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.240953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.125:41613 (local address 127.0.0.1:54884)
0805 22:46:53.241006 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.241017 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:53.241052 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:53.241184 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:53.241185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:53.241351 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:53.241411 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.241412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.242243 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.242245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.242958 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.242960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.243816 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.243817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.243906 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.244255 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:53.244259 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:53.245919 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:53.248374 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.248374 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.248375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.248716 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.248998 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.248998 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.248999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.249066 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.249371 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:53.249372 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:53.249416 (+ 44us) client_negotiation.cc:769] Sending connection context
0805 22:46:53.249489 (+ 73us) client_negotiation.cc:240] Negotiation successful
0805 22:46:53.249580 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260805 22:46:53.250113 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.241010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54884 (local address 127.23.197.125:41613)
0805 22:46:53.241058 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.241061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.241086 (+ 25us) server_negotiation.cc:407] Connection header received
0805 22:46:53.241111 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.241112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.241122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.241172 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.241500 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.242163 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.243044 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.243381 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.243459 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.246017 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.246019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.246019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.246027 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.247987 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.248832 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.248834 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.248834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.248922 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.249139 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.249140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.249140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.249289 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.249365 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.249925 (+ 560us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.250014 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260805 22:46:53.250283 25529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1: Bootstrap starting.
I20260805 22:46:53.251096 25529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:53.251411 25529 log.cc:824] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:53.251710 25745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.126@KRBTEST.COM
I20260805 22:46:53.252355 25529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1: No bootstrap required, opened a new log
I20260805 22:46:53.254086 25529 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } }
I20260805 22:46:53.254264 25529 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef3b68e7668d46a599a100023c3585e1, State: Initialized, Role: FOLLOWER
I20260805 22:46:53.254349 25529 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } }
I20260805 22:46:53.254736 25529 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:53.254662 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.255563 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:53.256642 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.247637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50378 (local address 127.23.197.126:42145)
0805 22:46:53.247678 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.247680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.247688 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:53.247746 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.247747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.247756 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.247811 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.248347 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.249105 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.250063 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.250337 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.250409 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.252799 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.252801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.252801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.252809 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.255051 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.255472 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.255473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.255473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.255538 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.255758 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.255759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.255759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.255886 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.256032 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.256459 (+ 427us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.256546 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970013, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.124@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
W20260805 22:46:53.257295 25771 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:53.257361 25771 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:53.259850 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.251981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41450)
0805 22:46:53.252056 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.252066 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:53.252105 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:53.252969 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:53.252970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:53.253115 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:53.253167 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.253168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.253772 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.253773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.254515 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.254518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.255029 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.255030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.255079 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.255629 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:53.255635 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:53.257059 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:53.258837 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.258838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.258838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.259068 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.259262 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.259263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.259263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.259313 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.259587 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:53.259588 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:53.259618 (+ 30us) client_negotiation.cc:769] Sending connection context
0805 22:46:53.259671 (+ 53us) client_negotiation.cc:240] Negotiation successful
0805 22:46:53.259752 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260805 22:46:53.260314 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.252029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.23.197.124:44413)
0805 22:46:53.252841 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.252844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.252853 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:53.252872 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.252873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.252883 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.252946 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.253245 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.253695 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.254609 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.254947 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.255031 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.257136 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.257137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.257137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.257143 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.258751 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.259152 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.259153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.259153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.259204 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.259366 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.259366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.259367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.259516 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.259568 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.260146 (+ 578us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.260222 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789}
I20260805 22:46:53.258519 25419 master_runner.cc:186] Error getting master registration for 127.23.197.125:41613: OK, instance_id { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" instance_seqno: 1785970013048902 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 22:46:53.261240 25419 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 22:46:53.261700 25745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee: Bootstrap starting.
I20260805 22:46:53.262298 25745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:53.262598 25745 log.cc:824] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:53.263429 25745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee: No bootstrap required, opened a new log
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.197.125@KRBTEST.COM
I20260805 22:46:53.265961 25745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } }
I20260805 22:46:53.266129 25745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258bb47e936343c9b14a4df3b38556ee, State: Initialized, Role: FOLLOWER
I20260805 22:46:53.266227 25745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } }
I20260805 22:46:53.266553 25745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:53.266522 25774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.266731 25774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:53.267814 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.257525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54896 (local address 127.23.197.125:41613)
0805 22:46:53.257561 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.257563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.257571 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:53.257616 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.257617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.257627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.257686 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.258337 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.258920 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.259910 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.260207 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.260270 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.264602 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.264604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.264604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.264613 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.266377 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.266875 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.266877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.266877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.266944 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.267187 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.267188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.267189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.267340 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.267516 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.267555 (+ 39us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.267705 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260805 22:46:53.269043 25788 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:53.269116 25788 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:53.269340 24341 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 22:46:53.270265 25489 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ef3b68e7668d46a599a100023c3585e1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50378
I20260805 22:46:53.270402 25489 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260805 22:46:53.270450 25489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:53.271464 25489 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } }
I20260805 22:46:53.271732 25489 leader_election.cc:288] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [CANDIDATE]: Term 1 election: Requested vote from peers 515b617d6c7e4e4ba160b3b162cd57e6 (127.23.197.125:41613), 258bb47e936343c9b14a4df3b38556ee (127.23.197.124:44413)
I20260805 22:46:53.272166 25709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef3b68e7668d46a599a100023c3585e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "258bb47e936343c9b14a4df3b38556ee"
I20260805 22:46:53.272289 25709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:53.270120 25653 master_runner.cc:186] Error getting master registration for 127.23.197.125:41613: OK, instance_id { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" instance_seqno: 1785970013048902 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 22:46:53.272534 25653 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 22:46:53.273244 25709 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef3b68e7668d46a599a100023c3585e1 in term 1.
I20260805 22:46:53.273463 25447 leader_election.cc:302] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 258bb47e936343c9b14a4df3b38556ee, ef3b68e7668d46a599a100023c3585e1; no voters:
I20260805 22:46:53.273877 25758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 22:46:53.274040 25758 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [term 1 LEADER]: Becoming Leader. State: Replica: ef3b68e7668d46a599a100023c3585e1, State: Running, Role: LEADER
I20260805 22:46:53.274135 25758 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } }
I20260805 22:46:53.274557 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef3b68e7668d46a599a100023c3585e1. Latest consensus state: current_term: 1 leader_uuid: "ef3b68e7668d46a599a100023c3585e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.274636 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:53.275107 25811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 22:46:53.275317 25811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 22:46:53.277319 25709 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [term 1 FOLLOWER]: Refusing update from remote peer ef3b68e7668d46a599a100023c3585e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:53.278192 25758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:53.279615 25774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef3b68e7668d46a599a100023c3585e1. Latest consensus state: current_term: 1 leader_uuid: "ef3b68e7668d46a599a100023c3585e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.279721 25774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:53.281777 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef3b68e7668d46a599a100023c3585e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.281811 25774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef3b68e7668d46a599a100023c3585e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.281869 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [sys.catalog]: This master's current role is: LEADER
I20260805 22:46:53.281894 25774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:53.281955 25813 mvcc.cc:205] Tried to move back new op lower bound from 7315333174380494848 to 7315333174371557376. Current Snapshot: MvccSnapshot[applied={T|T < 7315333174380494848}]
I20260805 22:46:53.282732 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.273679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41454 (local address 127.23.197.124:44413)
0805 22:46:53.273718 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.273720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.273926 (+ 206us) server_negotiation.cc:407] Connection header received
0805 22:46:53.274036 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.274037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.274047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.274097 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.274810 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.275527 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.276635 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.276945 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.276992 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.278939 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.278940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.278940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.278948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.280620 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.281574 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.281576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.281576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.281647 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.282016 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.282017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.282018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.282180 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.282275 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.282544 (+ 269us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.282643 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260805 22:46:53.284005 25812 mvcc.cc:205] Tried to move back new op lower bound from 7315333174380494848 to 7315333174371557376. Current Snapshot: MvccSnapshot[applied={T|T < 7315333174380494848}]
I20260805 22:46:53.284600 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.273914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50392 (local address 127.23.197.126:42145)
0805 22:46:53.273969 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.273971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.276254 (+ 2283us) server_negotiation.cc:407] Connection header received
0805 22:46:53.276358 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.276359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.276368 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.276398 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.277217 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.277909 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.278880 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.279125 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.279196 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.280492 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.280493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.280494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.280501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.282678 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.283563 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.283564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.283564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.283630 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.283934 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.283935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.283935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.284077 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.284191 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.284402 (+ 211us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.284495 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260805 22:46:53.287914 25641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } attempt: 3
I20260805 22:46:53.292479 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.274817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54898 (local address 127.23.197.125:41613)
0805 22:46:53.274976 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.274978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.282761 (+ 7783us) server_negotiation.cc:407] Connection header received
0805 22:46:53.283153 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.283154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.283167 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.283200 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.283821 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.284551 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.285656 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.285971 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.286047 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.287584 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.287586 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.287597 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.287608 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.289477 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.290815 (+ 1338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.290816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.290816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.290892 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.291361 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.291362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.291362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.291501 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.291597 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.292293 (+ 696us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.292368 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
Aug 05 22:46:53 dist-test-slave-g170 krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785970013, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.197.125@KRBTEST.COM for kudu/127.23.197.124@KRBTEST.COM
I20260805 22:46:53.293222 25811 catalog_manager.cc:1347] Generated new cluster ID: 8e23e221a3604d7182b0ece824eae748
I20260805 22:46:53.293277 25811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 22:46:53.295907 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.288290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.197.124:44413 (local address 127.0.0.1:41458)
0805 22:46:53.288339 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0805 22:46:53.288350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 22:46:53.288391 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 22:46:53.288582 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 22:46:53.288583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 22:46:53.288881 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0805 22:46:53.288928 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.288930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.289783 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.289784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.290565 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 22:46:53.290567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.291052 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.291053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 22:46:53.291126 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.291535 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 22:46:53.291540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0805 22:46:53.293022 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 22:46:53.294885 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.294886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.294886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.295089 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.295303 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 22:46:53.295303 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 22:46:53.295304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 22:46:53.295357 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 22:46:53.295634 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 22:46:53.295634 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 22:46:53.295672 (+ 38us) client_negotiation.cc:769] Sending connection context
0805 22:46:53.295729 (+ 57us) client_negotiation.cc:240] Negotiation successful
0805 22:46:53.295814 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260805 22:46:53.295935 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:53.288315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.23.197.124:44413)
0805 22:46:53.288371 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.288373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.288382 (+ 9us) server_negotiation.cc:407] Connection header received
0805 22:46:53.288467 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.288468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.288491 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.288540 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.289024 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.289708 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.290672 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:53.290977 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:53.291053 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:53.293119 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 22:46:53.293121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 22:46:53.293121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 22:46:53.293129 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 22:46:53.294805 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.295171 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.295172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.295172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.295242 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 22:46:53.295428 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 22:46:53.295429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 22:46:53.295430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 22:46:53.295566 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 22:46:53.295633 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:53.295769 (+ 136us) server_negotiation.cc:299] Negotiation successful
0805 22:46:53.295847 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260805 22:46:53.298154 25641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6: Bootstrap starting.
I20260805 22:46:53.299031 25641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6: Neither blocks nor log segments found. Creating new log.
I20260805 22:46:53.299378 25641 log.cc:824] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6: Log is configured to *not* fsync() on all Append() calls
I20260805 22:46:53.317541 25641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6: No bootstrap required, opened a new log
I20260805 22:46:53.318673 25811 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 22:46:53.319612 25811 catalog_manager.cc:1504] Loading token signing keys...
I20260805 22:46:53.319849 25641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } }
I20260805 22:46:53.320010 25641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 515b617d6c7e4e4ba160b3b162cd57e6, State: Initialized, Role: FOLLOWER
I20260805 22:46:53.320093 25641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } }
I20260805 22:46:53.320464 25641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 22:46:53.320586 25604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef3b68e7668d46a599a100023c3585e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "515b617d6c7e4e4ba160b3b162cd57e6"
I20260805 22:46:53.320802 25604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 22:46:53.321856 25604 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef3b68e7668d46a599a100023c3585e1 in term 1.
I20260805 22:46:53.322049 25822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.322135 25822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:53.322398 25602 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [term 1 FOLLOWER]: Refusing update from remote peer ef3b68e7668d46a599a100023c3585e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 22:46:53.325450 25808 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 22:46:53.328184 25822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef3b68e7668d46a599a100023c3585e1. Latest consensus state: current_term: 1 leader_uuid: "ef3b68e7668d46a599a100023c3585e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.328297 25822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [sys.catalog]: This master's current role is: FOLLOWER
W20260805 22:46:53.328321 25842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 22:46:53.328400 25842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 22:46:53.328331 25822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef3b68e7668d46a599a100023c3585e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.328532 25822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 22:46:53.330297 25811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1: Generated new TSK 0
I20260805 22:46:53.330551 25811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 22:46:53.330008 25845 mvcc.cc:205] Tried to move back new op lower bound from 7315333174544429056 to 7315333174371557376. Current Snapshot: MvccSnapshot[applied={T|T < 7315333174380494848}]
I20260805 22:46:53.331151 25819 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef3b68e7668d46a599a100023c3585e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3b68e7668d46a599a100023c3585e1" member_type: VOTER last_known_addr { host: "127.23.197.126" port: 42145 } } peers { permanent_uuid: "515b617d6c7e4e4ba160b3b162cd57e6" member_type: VOTER last_known_addr { host: "127.23.197.125" port: 41613 } } peers { permanent_uuid: "258bb47e936343c9b14a4df3b38556ee" member_type: VOTER last_known_addr { host: "127.23.197.124" port: 44413 } } }
I20260805 22:46:53.331259 25819 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef3b68e7668d46a599a100023c3585e1 [sys.catalog]: This master's current role is: LEADER
W20260805 22:46:53.356958 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.354978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.23.197.124:44413)
0805 22:46:53.355028 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.355031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.356135 (+ 1104us) server_negotiation.cc:407] Connection header received
0805 22:46:53.356257 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.356259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.356271 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.356306 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.356880 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41460: BlockingRecv error: recv got EOF from 127.0.0.1:41460 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260805 22:46:53.357841 25536 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.355442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50400 (local address 127.23.197.126:42145)
0805 22:46:53.355487 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.355489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.357251 (+ 1762us) server_negotiation.cc:407] Connection header received
0805 22:46:53.357363 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.357364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.357376 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.357409 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.357745 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50400: BlockingRecv error: recv got EOF from 127.0.0.1:50400 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260805 22:46:53.358652 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.355836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54912 (local address 127.23.197.125:41613)
0805 22:46:53.355880 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.355882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.358079 (+ 2197us) server_negotiation.cc:407] Connection header received
0805 22:46:53.358186 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.358187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.358198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.358230 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.358588 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54912: BlockingRecv error: recv got EOF from 127.0.0.1:54912 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260805 22:46:53.361253 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.360429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.23.197.125:41613)
0805 22:46:53.360467 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.360469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.360654 (+ 185us) server_negotiation.cc:407] Connection header received
0805 22:46:53.360800 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.360801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.360821 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.360856 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.361205 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54926: BlockingRecv error: recv got EOF from 127.0.0.1:54926 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260805 22:46:53.364439 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.363624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54940 (local address 127.23.197.125:41613)
0805 22:46:53.363660 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.363662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.363897 (+ 235us) server_negotiation.cc:407] Connection header received
0805 22:46:53.363998 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.363999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.364010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.364041 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.364395 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54940: BlockingRecv error: recv got EOF from 127.0.0.1:54940 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260805 22:46:53.368566 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.367776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.23.197.125:41613)
0805 22:46:53.367812 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.367814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.368040 (+ 226us) server_negotiation.cc:407] Connection header received
0805 22:46:53.368142 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.368143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.368153 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.368184 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.368518 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54948: BlockingRecv error: recv got EOF from 127.0.0.1:54948 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260805 22:46:53.374629 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.373816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54950 (local address 127.23.197.125:41613)
0805 22:46:53.373854 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.373855 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.374084 (+ 229us) server_negotiation.cc:407] Connection header received
0805 22:46:53.374194 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.374195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.374205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.374237 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.374581 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54950: BlockingRecv error: recv got EOF from 127.0.0.1:54950 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260805 22:46:53.384974 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.384035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54956 (local address 127.23.197.125:41613)
0805 22:46:53.384085 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.384087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.384339 (+ 252us) server_negotiation.cc:407] Connection header received
0805 22:46:53.384442 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.384444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.384455 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.384484 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.384904 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54956: BlockingRecv error: recv got EOF from 127.0.0.1:54956 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260805 22:46:53.403644 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.402497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54964 (local address 127.23.197.125:41613)
0805 22:46:53.402554 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.402556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.402930 (+ 374us) server_negotiation.cc:407] Connection header received
0805 22:46:53.403045 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.403046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.403060 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.403092 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.403586 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54964: BlockingRecv error: recv got EOF from 127.0.0.1:54964 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260805 22:46:53.438215 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.437204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.23.197.125:41613)
0805 22:46:53.437251 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.437255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.437542 (+ 287us) server_negotiation.cc:407] Connection header received
0805 22:46:53.437655 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.437656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.437670 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.437702 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.438150 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54974: BlockingRecv error: recv got EOF from 127.0.0.1:54974 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260805 22:46:53.504565 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.503762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.23.197.125:41613)
0805 22:46:53.503836 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.503839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.503971 (+ 132us) server_negotiation.cc:407] Connection header received
0805 22:46:53.504049 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.504050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.504063 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.504121 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.504499 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54982: BlockingRecv error: recv got EOF from 127.0.0.1:54982 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260805 22:46:53.634793 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.633893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54986 (local address 127.23.197.125:41613)
0805 22:46:53.633972 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.633974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.634106 (+ 132us) server_negotiation.cc:407] Connection header received
0805 22:46:53.634161 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.634162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.634175 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.634232 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.634716 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54986: BlockingRecv error: recv got EOF from 127.0.0.1:54986 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260805 22:46:53.892937 25649 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 22:46:53.892147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.23.197.125:41613)
0805 22:46:53.892228 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:53.892230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:53.892353 (+ 123us) server_negotiation.cc:407] Connection header received
0805 22:46:53.892390 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:53.892391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:53.892404 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:53.892461 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0805 22:46:53.892870 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54992: BlockingRecv error: recv got EOF from 127.0.0.1:54992 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 22:46:54.269714 25788 catalog_manager.cc:1259] Loaded cluster ID: 8e23e221a3604d7182b0ece824eae748
I20260805 22:46:54.269829 25788 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee: loading cluster ID for follower catalog manager: success
I20260805 22:46:54.271364 25788 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee: acquiring CA information for follower catalog manager: success
I20260805 22:46:54.271836 25788 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 258bb47e936343c9b14a4df3b38556ee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 22:46:54.328902 25842 catalog_manager.cc:1259] Loaded cluster ID: 8e23e221a3604d7182b0ece824eae748
I20260805 22:46:54.328975 25842 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6: loading cluster ID for follower catalog manager: success
I20260805 22:46:54.329747 25842 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6: acquiring CA information for follower catalog manager: success
I20260805 22:46:54.329993 25842 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 515b617d6c7e4e4ba160b3b162cd57e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 22:46:54.409147 25947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:54.406289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.23.197.125:41613)
0805 22:46:54.406417 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:54.406419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:54.406535 (+ 116us) server_negotiation.cc:407] Connection header received
0805 22:46:54.406587 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:54.406588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:54.406647 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:54.406735 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:54.406890 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.407400 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.408203 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.408397 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.408451 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:54.408602 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:54.408726 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:54.408867 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:54.408925 (+ 58us) server_negotiation.cc:299] Negotiation successful
0805 22:46:54.408949 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260805 22:46:54.413648 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:54.411111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.23.197.124:44413)
0805 22:46:54.411234 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:54.411236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:54.411244 (+ 8us) server_negotiation.cc:407] Connection header received
0805 22:46:54.411318 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:54.411318 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:54.411353 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:54.411417 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:54.411516 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.412025 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.412872 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.413065 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.413119 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:54.413280 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:54.413373 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:54.413436 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:54.413495 (+ 59us) server_negotiation.cc:299] Negotiation successful
0805 22:46:54.413515 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260805 22:46:54.418813 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:54.415590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41480 (local address 127.23.197.124:44413)
0805 22:46:54.415645 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:54.415647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:54.415783 (+ 136us) server_negotiation.cc:407] Connection header received
0805 22:46:54.415888 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:54.415889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:54.415901 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:54.415968 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:54.416098 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.416967 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.417920 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.418214 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.418279 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:54.418490 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:54.418525 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:54.418581 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:54.418701 (+ 120us) server_negotiation.cc:299] Negotiation successful
0805 22:46:54.418733 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260805 22:46:54.418819 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:54.415497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.23.197.126:42145)
0805 22:46:54.415598 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:54.415600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:54.415669 (+ 69us) server_negotiation.cc:407] Connection header received
0805 22:46:54.415790 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:54.415791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:54.415847 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:54.415902 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:54.416143 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.416663 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.417710 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.418085 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.418193 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:54.418444 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:54.418556 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:54.418645 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:54.418707 (+ 62us) server_negotiation.cc:299] Negotiation successful
0805 22:46:54.418735 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":46,"threads_started":1}
I20260805 22:46:54.418861 25947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 22:46:54.415680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.23.197.125:41613)
0805 22:46:54.415734 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0805 22:46:54.415737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 22:46:54.415887 (+ 150us) server_negotiation.cc:407] Connection header received
0805 22:46:54.416004 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 22:46:54.416005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 22:46:54.416017 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 22:46:54.416050 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 22:46:54.416276 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.417101 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.418010 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 22:46:54.418316 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 22:46:54.418408 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 22:46:54.418580 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 22:46:54.418611 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 22:46:54.418650 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0805 22:46:54.418761 (+ 111us) server_negotiation.cc:299] Negotiation successful
0805 22:46:54.418784 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260805 22:46:54.420958 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 25419
I20260805 22:46:54.436527 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 25540
I20260805 22:46:54.449975 24341 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHuFw91/build/release/bin/kudu with pid 25653
2026-08-05T22:46:54Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3084 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6177 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (289366 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST